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ABSTRACT 

Noise  generation  caused  by  quantization  of  signals  In 
digital  filters  Is  considered  using  matrix  state-space  tech- 
niques.  The  results  are  expressed  In  terms  of  the  correla- 
tion of  the  errors  so  that  the  assumptions  made  about  the 
statistical  properties  of  the  noise  sources  can  be  evaluated. 
The  effect  of  correlation,  which  is  shown  to  be  significant, 
depends  upon  the  specific  structure  of  the  filter  and  the 
relative  location  of  the  error  source  both  topologically  and 
temporally. 

Specifically,  the  results  are  applied  to  a  newly  extended 
set  of  canonic  realizations  of  second  order  filters  which  are 
presented  in  the  form  of  canonic  arrays  of  coefficients. 
Output  noise  formulae  for  these  realizations  are  presented 
for  the  two  extreme  assumptions  of  uncorrelated  error  sources 

and  constant  noise  generation  and  are  confirmed  by  experi- 
ment.  These  formulae  are  directly  useful  in  analysis,  design, 
and  comparison. 

Additionally,  new  algorithms  are  presented,  one  for 
performing  circular  convolutions  and  the  other  for  calculating 
the  Discrete  Fourier  Transform. 


TABLE  OF  CONTENTS 

I.  INTRODUCTION ll| 

A.  IMPORTANCE  OF  DIGITAL  SIGNAL  PROCESSING  14 

B.  AREA  OP  INVESTIGATION 111 

C.  PREVIEW  OF  RESULTS 16 

II.  DIGITAL  SIGNAL  PROCESSING  I9 

A.   DISCRETE  TIME  PROCESSING I9 

1.  Z-Transform I9 

a.  Definition 19 

b.  Relation  to  Laplace-Fourier 
Transforms 20 

2.  Functional  Transforms  21 

a.  Standard  (Impulse  Invcrlant) 
Z-Transform 23 

b.  Bilinear  Z-Transform  23 

c.  Matched  Z-Transform 2k 

d.  Higher-Order  Transforms  24 

e.  Finite  Impulse  Response  (FIR) 

Filters 25 

3.  Transfer  Function  Realizations  26 

a.  Definitive  Forms  26 

b.  Reduction  to  Lower  Order  Forms  27 

(1)  Parallel  Realization  by  Partial 
Fraction  Expansion  28 

(2)  Cascade  Realization  by 
Factorization  28 

(3)  Hybrid  Realization  28 


c.  Canonic  First  and  Second 

Order  Sections 28 

d.  Other  Forms  of  Realizations  29 

4.  Discrete  Fourier  Transform  29 

5.  Other  Transform  Techniques  31 

B.   DIGITAL  PROCESSING  31 

1.  A/D  Conversion 31 

2.  The  Approximation  Problem 32 

3.  Quantization  Errors  33 

III.  CANONIC  REALIZATIONS  OF  SECOND-ORDER  SECTIONS  -  ^2 

A.  INTRODUCTION ^2 

B.  TRANSFER  FUNCTIONS  ^3 

1.  Basic  Forms ^3 

2.  Realizability ^S 

C.  STATE  EQUATIONS 50 

1.   Basic  Forms 50 

D.  COEFFICIENTS  OF  STATE  EQUATIONS  58 

E.  CANONIC  ARRAYS 63 

F.  CONCLUSIONS 75 

IV.  ERROR  ANALYSIS  OF  RECURSIVE  DIGITAL  FILTERS  1^2 

A.  INTRODUCTION 1^2 

B.  STATE  SPACE  FORMULATION  1^3 

C.  STOCHASTIC  FORMULATION  WITH  CORRELATION  1^9 

1.  General  Formulation  1^9 

a.   Structural  Effects  (General)  159 

2.  Simplification  to  General  Second- 
Order  Section 162 


a.  For  Arbitrary  State  Equations  162 

b.  For  Canonic  Arrays  163 

c.  Uncorrelated  Case 166 

d.  Correlated  Case 176 

(1)  Reduction  for  Uncorrelated 

Input  183 

(2)  Reduction  for  Highly 

Correlated  Input  I85 

D.   CONCLUSIONS 191 

DISCRETE  FOURIER  TRANSFORM  FILTER  198 


A. 
B. 
C. 


D. 

E. 
F. 

APPENDIX  A 
APPENDIX  B 
APPENDIX  C 

APPENDIX  D: 


INTRODUCTION 198 

DEFINITION  OF  TERMS  199 

THE  DFT  FILTER 200 

1.  Formulation 200 

2.  Response  to  a  Finite  Duration  Input  2C3 

3.  State  Analysis 20? 

a.  Example  1:   Unit  Pulse  Response 211 

b.  Example  2:   Response  to  a  Periodic 

Input 2li| 

'^.   Output  Analysis  21^ 

MODIFIED  DFT  FILTER  219 

A  NEW  DFT  ALGORITHM 222 

CONCLUSIONS  237 

CORRELATION  IN  LIMIT  CYCLES  264 

EXPERIMENTS  IN  ERROR  CROSS-CORRELATION 270 


VERIFICATION  OF  POSITIVITY  OF  FACTORS 
IN  THE  OUTPUT  ERROR  VARIANCE  


325 


EXPERIMENTS  IN  ERROR  PROPAGATION 

IN  SELECTED  DIGITAL  FILTERS 33^ 


APPENDIX  E 
APPENDIX  F 
APPENDIX   G 

APPENDIX  H; 


SPECTRAL   THEORY   IN   DIGITAL   FILTERS   370 

SIGNAL   CORRELATIONS   FOR  A   TM ^     FILTER  577 

00  -"  ' 

rpTT 

TRANSFORMING  (M)   -ORDER  COMPLEX  FILTERS 

INTO  (2M)TH_0RDER  REAL  FILTERS  392 


COMPLEX   INTEGRATION   INVOLVING   A 
COMPLEX   DIRAC   DELTA   FUNCTION   — 


■399 


LIST  OF  REFERENCES il01 

INITIAL   DISTRIBUTION   LIST   ^06 


LIST  OF  TABLES 


Table  2-1 

Table  3-1 

Table  3- 2 a 

Table  3-2b 

Table  3-3a 

Table  3-3b 

Table  4-1 

Table  ^-2 

Table  D-1 
Table  D-2 

Table  D-3 


Present  Bounds  on  Quantization  Error ijy 

State  and  Output  Equation  Sets 55 

Canonic  Arrays,  SM ,  .  67 

Canonic  Arrays,  SN..  68 

Canonic  Arrays,  TM.  69 

Canonic  Array,  TNq 69 

2 
Formulae  for  a.   for  Uncorrelated, 

Identically  Distributed  Error  Sources  174 

Maximum  Mean-Squared  Output  Error  for 
Hypothesized  Limit  Cycle  Analysis 192 

Parameters  of  TMqq  Filter  Experiments 3^5 

Results  of  TMqq  Filter  Experim.ent  for 

Random  Input 346 

Results  of  TMqq  Filter  Experiment  for 

Step  Inputs 3^17 


LIST  OF  FIGURES 

Figure  2-1  Direct  Realization  of  N^^  Order 

Transfer  Function  37 

Figure  2-2   Canonic  Realization  of  N^^  Order 

Transfer  Function  38 

Figure  2-3  Parallel  Realization  of  H(z) 39 

Figure  2-H      Cascade  Realization  of  H(z)  39 

Figure  2-5  Hybrid  Realizations  of  H(z)  kO 

Figure  2-6a  Canonic  First  Order  Section  4l 

Figure  2-6b  Canonic  Second  Order  Section  Hi 

Figures  3-1  through  3-9  Realizations  of  SM^ .  77-9^ 

Figures  3-10  through  2-l8  Realizations  of  SN. .  95-112 

Figures  3-19  through  3-21  Realizations  of  TM^^ 113-118 

Figure  3-22  Realizations  of  TNq 119 

Figure  2-23  Equivalent  Symbols  for  Digital  Switches  —  121 

Figures  3-24  through  3-32  Realizations  of  SM^^  122-126 

Figures  3-33  through  3-^1  Realizations  of  SN^j  127-131 

Figures  3-^2  through  3-50  Realizations  of  TM^t^ 132-136 

Figures  3-5l  through  3-59   Realizations  of  TN.^  137-1^1 

Figure  4-la  Realization  of  TM^q  Including 

Quantization  Error  Sources  196 

Figure  H-lt   Realization  of  TMqq  Including 

Composite  Error  Sources  196 

Figure  '<-2a  Nonllnearity  of  Rounding 197 

Figure  i»-2b  Error  Function  for  Rounding 197 

Figure  5-1   DFT  FILTER 239 

Figure  5-2   Convolution  of  Two  Sequences  of  Length  N  -  2^0 


10 


Figure  5-3  Convolution  of  a  Length-N  Sequence 
with  the  Single  Period  Extension 
of  Another  Length-N  Sequence  241 

Figure  5-4   Definition  of  Sequence  {u  (n')}  for 

N=4 242 

Figure  5-5  State  Unit  Pulse  Response  of  DFT 

FILTER  for  N=4 243 

Figure  5-6   Response  of  DFT  FILTER  to  a  Periodic 

Input  for  N=4 244 

Figure  5-7  Circular  Convolution  Generator 

in  DFT  Form 245 

Figure  5-8  Circular  Convolution  Generator 

in  FIR  Form 246 

Figure  5-9  Availability  Time  of  Convolution  of 

Sequence  (u  (m)}  for  N=8 24? 

Figure  5-10  Circular  Convolution  Generator 

(Definitive  Form)  248 

Figure  5-11  Rearrangement  of  Circular  Convolution 

Generator  of  Figure  5-10 249 

Figure  5-12  Rearrangement  of  Circular  Convolution 

Generator  of  Figure  5-8 250 

Figure  5-13  Modified  DFT  FILTER 251 

Figure  5-l4  Modified  Circular  Convolution  Generator  -  252 

Figure  5-15  Availability  Time  of  Convolution 

Sequence  {u  (m)}  for  N=8  in  Modified 
Generator 253 

Figure  5-16  DFT  Algorithm  for  N=8 254 

Figure  5-17  Flow  Graph  Manipulations  of  DFT 

Algorithm 255 

Figure  5-l8  Rearrangement  of  DFT  Algorithm  of 

Figure  5-l6 256 

Figure  5-19  Four-Delay  DFT  Stages  257 

Figure  5-20  Four-Delay  DFT  Stages  with  Minimum 

Non-Trivial  Multiplications  258 


11 


Figure  5-21  Eight-Delay  DFT  Stage  with 

Feedback  Coefficient,  J  259 

Figure  5-22  Eight-Delay  J-Feedback  DFT  Stage  with 

Minimum  Non-Trivial  Multiplication  260 

Figure  5-23  Sixteen-Delay  J-Feedback  DFT  Stage  26l 

Figure  5-24  Sixteen-Delay  j-Feedback  DFT  Stage  with 

Minimum  Non-Trivial  Multiplications  262 

Figure  5-25  Parallel  Input  DPT  Algorithm  Equivalent 

to  Figure  5-l8 263 

Figure  A-la  Case  1  Limit  Cycle 268 

Figure  A-lb  Case  0  Limit  Cycle 268 

Figure  A-lc  Case  2  Limit  Cycle 268 

Figure  A-2   Correlation  in  Limit  Cycles  269 

Figure  B-1  Quantization  Error  Simulation  Model  280 

Figure  B-2  Error  Functions  for  R  =  Aj/A^  =  3/2  280 

Figures  B-3a  through  B-3d  Error  Distribution?  281-284 

Figures  B-4  through  B-8  Correlation  Coefficients  285-288 

Figures  B-9.1a  through  B-9.22  Error  Distributions  —  289-324 

Figure  C-1  Sign  of  A  in  the  (a,b)-Plane 333 

Figure  C-2  Plot  of  (l+b)/A  in  the  Stable  Region  333 

Figure  D-1   The  Driving  Function  Autocorrelation  348 

Figures  D-2a  through  D-2g  Responses  of  Filter  0  349-355 

Figures  D-3a  through  D-3g  Responses  of  Filter  1  356-362 

Figures  D-4a  through  D-4g  Responses  of  Filter  2  363-369 

Figure  G-1   First  Order  Complex  Coefficient 

Digital  Filter  398 

Figure  G-2   Second  Order  Real  Coefficient 

Digital  Filter  398 


12 


ACKNOWLEDGMENT 

I  would  like  to  express  my  appreciation: 

To  Professor  Sydney  R.  Parker  for  his  guidance  and  motivation; 

To  my  wife,  Diana,  for  her  encouragement  and  perseverance; 

To  my  typist,  Rosemary  Lande,for  her  efficient,  cooperative 
efforts  to  assist  me  in  completing  this  work  on  time; 

And  to  the  United  States  Navy  for  allowing  me  the  tlm.e  to 
pursue  this  goal. 


13 


I.   INTRODUCTION 

A.  IMPORTANCE  OF  DIGITAL  SIGNAL  PROCESSING 

One  of  the  most  pervasive  advances  in  technology  today 
has  been  in  the  area  of  digital  signal  processing.   Since 
the  time  when  the  transistor  made  the  proliferation  of  general 
purpose  digital  computers  economically  feasible,  other  ad- 
vances, particularly  in  integrated  circuit  technology,  made 
practical  the  development  of  dedicated  digital  processors. 
From  the  common  telephone  and  television,  to  rapid  transit 
systems,  manufacturing  processes,  speech  therapy,  and  the 
exploration  of  space,  digital  signal  processors  have  had 
their  Impact.   In  the  Navy,  digital  signal  processors  are 
becoming  more  and  more  widely  used  in  radar;  sonar;  missile 
and  torpedo  guidance;  launcher  control  system.s;  combat 
Information  collection,  dissemination  and  display;  and  of 
course  communications  systems. 

B.  AREA  OP  INVESTIGATION 

A  digital  filter  is  defined  [1]  as  a  computational  pro- 
cess or  algorithm  by  which  a  digital  (discrete-time  and 
discrete-value)  signal  or  sequence  of  numbers  termed  the 
input  is  transformed  into  an  output  digital  signal.   A 
digital  filter  may  be  Implemented  as  a  subroutine  in  a 
general  purpose  computer  or  as  hardware  in  the  form  of  a 
special  purpose  digital  processor. 


Ih 


Digital  filters  are  a  subclass  of  the  set  of  discrete- 
time  filters  which  also  Includes  "sampled  data"  filters  In 
which  the  signals  may  take  on  a  continuum  of  values.   The 
mathematical  treatment  of  the  discrete-time  nature  of  these 
filters  is  common  to  both  digital  and  sampled  data  filters. 
But  in  digital  filters  the  discrete  value  property  causes 
deleterious  effects  in  the  processing  of  signals. 

The  reason  that  digital  filters  must  operate  on  discrete 
valued  signals  is  that  there  is  only  a  finite  number  of 
digits  available  with  which  to  represent  the  signal  values. 
The  only  way  to  transform  one  set  of  numbers  into  another 
set  of  numbers  is  by  a  combination  of  the  operations  of 
multiplication  and  addition  and  it  is  also  necessary  in  a 
digital  filter  that  multiplier  coefficients  be  representable 
in  a  finite  number  of  digits.   In  the  process  of  performing 
a  multiplication,  the  length  of  the  mantissa  in  the  represen- 
tation of  the  product  can  be  as  large  as  the  sum  of  the 
lengths  of  the  mantissas  of  the  m.ultipller  and  multiplicand. 
But  in  general  that  means  the  product  is  not  representable 
by  the  number  of  digits  available.   Therefore,  some  method 
of  quantizing  must  be  performed  during  each  sample  time  in 
order  to  maintain  the  representabllity  of  the  signal  quan- 
tities.  This  causes  non-linear  effects  even  In  a  filter 
which,  when  viewed  only  as  a  discrete-time  filter,  may 
appear  to  be  linear  in  nature . 

Many  authors  have  claim.ed  that  the  errors  generated  by 
the  quantization  performed  in  the  filter  are  uncorrelated 


15 


or  at  most  negligibly  correlated  from  sample-time  to  sample- 
time  and  between  the  sources  of  error.   But  when  the  phe- 
nomenon of  limit  cycles  is  considered  it  becomes  obvious 
[Appendix  A]  that  there  is  a  great  deal  of  correlation 
involved.   Since  the  amount  of  correlation  depends  on  the 
value  of  the  coefficients  [Appendix  B]  and  the  method  of 
quantization,  it  follows  that  if  there  are  more  than  one 
structure  (algorithm)  by  which  a  particular  filter  may  be 
realized  and  the  values  of  the  coefficients  and  the  method 
of  quantization  are  different  between  these  structures,  then 
the  errors  generated  and  their  correlation  will  be  different. 

The  objective  of  this  work  Is  two-fold:   (1)  To  formulate 
a  general  approach  for  investigating  noise  generation  and 
propagation  to  include  correlation  and  structural  effects; 
and  (2)  to  apply  this  approach  to  specific  examples,  partic- 
ularly the  class  of  canonic  second  order  filter  structures 
which  are  derived  herein. 

C.   PREVIEW  OF  RESULTS 

After  a  summary  of  discrete  time  signal  processing  theory 
in  Chapter  II,  an  exhaustive  search  is  made  in  Chapter  III 
for  all  possible  realizations  of  a  second  order  digital 
filter  in  state  variable  form  which  are  considered  best  in 
a  certain  sense.   This  search  culminates  In  the  conclusion 
that  there  are  58  ways  to  realize  a  given  transfer  function 
v;hlch  meet  the  specified  criteria.   Thirty-eight  of  these 
realizations  have  not  appeared  previously  in  the  literature. 
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In  Chapter  IV,  following  a  method  presented  by  Parker 
and  Yakowltz  [2,3],  but  without  assuming  uncorrelated  errors, 
a  general  expression  for  the  output  error  variance  for  any 
order  filter  Is  derived.   This  expression  Is  then  applied 
to  the  second  order  case  and  then  to  the  realizations  pre- 
sented In  Chapter  III.   When  the  assumptions  of  uncorrelated 
and  highly  correlated  noise  are  made,  the  expressions  for 
the  output  error  variance  reduce  to  two  sets  of  formulae 
for  these  extremal  cases.   Examination  of  these  form.ulae 
reveals  for  the  first  time  the  effects  of  structure  on  noise 
generation  due  to  quantization.   Appendix  D  verifies  that 
these  formulae  yield  an  accurate  expression  for  the  output 
error  properties. 

The  quantization  noise  problem  is  particularly  serious 
in  recursive  digital  filters  wherein  the  algorithm  uses  the 
results  of  previous  calculations  to  generate  present  signal 
quantities.   The  fact  that  quantization  errors  are  also  fed 
back  is  the  root  cause  of  effects  such  as  limit  cycle  genera- 
tion.  On  the  other  hand,  non-recursive  filters  do  not  exhibit 
this  effect;  nor  do  recursive  filters  vjhich  produce  no  quan- 
tization errors  which  are  fed  back.   In  examining  the  nature 
of  non-recursive  and  recursive  finite  impulse  response  (FIR) 
dlscrete-tlm.e  filters  in  Chapter  V,  some  interesting  proper- 
ties of  the  signal  quantities  and  algorithms  are  observed. 
These  observations  lead  to  an  algorithm  for  computing  con- 
volutions which  is  most  efficient  (in  terms  of  number  of 
operations  performed)  when  completely  overlapping  blocks  of 
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data  from  a  continual  data  stream  need  to  be  processed  In 
this  way.   Additionally  a  new  method  for  computing  the  Dis- 
crete Fourier  Transform  (DFT)  is  presented  which  has  advan- 
tages over  other  methods  when  a  continual  data  stream  is  to 
be  processed.   These  algorithms  are  performed  without 
recursive  error  generation. 
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II.   DIGITAL  SIGNAL  PROCESSING 

A.   DISCRETE-TIME  PROCESSING"'- 

The  most  useful  and  most  used  tool  in  discrete-time 
signal  processing  is  the  Z-Transform.   Even  the  Discrete 
Fourier  Transform  is  a  special  case  of  the  Z-Transform. 
This  section  reviews  the  applications  of  the  Z-Transform, 

while  the  next  section  deals  with  the  problems  which  arise 

2 
when  this  theory  is  applied  to  digital  signal  processing. 

1.   The  Z-Transform 

a.   Definition 

CO 

Given  a  sequence  {x(n)}  __^  the  Z-Transform  of 
the  sequence  is  defined  as 


00 

X(z)  =  Z[x(n)]  =  I      x(n)z"^  (2.1) 

n=-~ 


This  definition  is  referred  to  as  the  two-sided  Z-Transform. 
Another  form  of  the  Z-Transform  is  defined  when  x(n)  =  0 
for  n  <  0.   Then  (2.1)  becomes 


X(z)  =   Z  x(n)z"^  (2.2) 

n=0 


The  use  of  the  phrase  discrete-time  rather  than  digital 
is  to  emphasize  that  the  theory  in  this  section  applies  to 
"sampled  data"  signal  processing,  in  which  the  data  can  take 
on  a  continuum  of  values,  as  well  as  digital  signal  process- 
ing, in  which  the  data  is  restricted  to  discrete  values  [1]. 

The  object  is  not  to  reiterate  known  theory  but  rather 
to  establish  notation  and  definitions  for  use  in  later  sections 
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This  definition  is  called  the  one-sided  Z-Transform.   Since 
(2.1)  is  more  general,  the  term  Z-Transform  will  refer  to 
(2.1)  unless  otherwise  specified. 

Both  (2.1)  and  (2.2)  can  be  viewed  as  Laurent 
series  in  the  complex  variable  z.   It  can  be  shown  [4]  that 
for  physical  systems  of  interest  (2.1)  and  (2.2)  converge 
on  the  unit  circle  in  the  z-plane.   Therefore,  since  the 
coefficients  of  a  Laurent  series  are  given  by  the  integral 


^'"'  =  5iJ  "C  «->^"  f  (2-3) 


where  C  is  a  closed  contour  in  the  region  of  convergence 
and  encircling  the  origin,  (2.3)  is  taken  to  be  the  inverse 
Z-Transform  and  C  Is  taken  to  be  the  unit  circle. 

b.   Relation  to  the  Laplace-Fourier  Transform 

When  the  sequence  {x(n)}  is  the  result  of  the 
equally  spaced  sampling  of  a  plecewlse  continuous  signal 
then  the  Z-Transform,  X(z)  Is  related  to  the  two-sided 
Laplace  Transform,  X(s),  of  the  function,  x(t),  by 


X(z) 


^^  =  I   Z   X(s  +  i^)  (2.i|) 

z=e       i<:=-~ 


v;here  j  =  ^ -1  and  T  is  the  spacing  between  samples.  Note 
that  this  relationship  is  not  one-to-one  since  there  are  an 
infinite  number  of  functions  x(t)  which  can  yield  the  same 

set  of  samples  x(n) .   Mathematically,  the  mapping  of  the 

sT 
s-plane  to  the  z-plane  by  z  =  e   is  many-to-one.   Since 
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-1    -sT   -1 
z   =  e    ,  z   Is  called  the  unit  delay  operator  by  the 

shifting  theorem  of  Laplace  Transform  theory. 

Since  the  Fourier  Transform  can  be  equivalently 

viewed  as  the  two-sided  Laplace  Transform  evaluated  at  s  =  ju) 

then  the  Z-Transform  evaluated  on  the  unit  circle  is  related 

to  the  Fourier  Transform  by 


X(z) 


00  .  00 

1      V      C/w    .STrkvs         1 


=   ±     E      X(j(a)+^))=i     I      X(j(a)+ka3    )) 


(2.5) 


where  oj  is  the  sampling  frequency  in  radians.   Notice  that 
H(e''   )  is  periodic  in  co  with  period  co  . 
2.   Functional  Transforms 

In  continuous  linear  theory,  the  relationship  between 
the  input,  u(t),  and  the  output,  v(t),is  given  by 

V(s)  =  H(s)U(s)  (2.6a) 


and 


00  00 

v(t)  =    /  h(T)u(t-T)dT  =    /  h(t-T)u(T)dT     (2.6b) 
_oo  _oo 


where  V(s)  and  U(s)  are  the  Laplace  Transforms  of  v(t)  and 
u(t)  respectively,  H(s)  is  the  Laplace  Transform  of  h(t) . 
The  function  h(t)  is  called  the  impulse  response  of  the 
filter  or  equivalently  the  Green's  function  of  the  linear 
Integro-differential  operator  which  relates  v(t)  to  u(t). 
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Equation  (2.6b)  Is  the  convolution  of  h(t)  with  u(t)  denoted 
by  h(t)*u(t).   H(s)  Is  called  the  transfer  function  or  fre- 
quency response  of  the  filter. 

In  discrete-time  linear  filter  theory,  the 
analogous  equations  hold.  I.e. 

V(z)  =  H(z)U(z)  (2.7a) 


00  00 

v(n)  =  I   h(m)u(n-m)  =   Z  h(n-m)u(m)        (2.7b) 
m=-~  m=-'» 


where  V(z),  H(z)  and  U(z)  are  the  Z-Trans forms  of  v(n), 
h(n)  and  u(n)  respectively.   The  sequence{h(n) }  Is  called 
the  Impulse  or  unit  pulse  response  and  H(z)  is  the  transfer 
function  of  the  filter. 

To  be  physically  realizable  h(t)  and  h(n)  must  be 
zero  for  t  <  0  and  n  <  0  respectively.   This  realizabllity 
condition  is  necessary  in  order  that  the  filter  not  react 
to  something  which  has  not  yet  occurred.   Thus  this  condition 
is  referred  to  as  causality. 

The  transfer  function,  H(z),  can  always  be  written 
as  a  ratio  of  polynomials  in  z,  and  H(s)  as  a  ratio  of  poly- 
nomials in  s  . 

In  designing  a  discrete-time  filter,  it  is  often 
the  case  that  one  tries  to  duplicate  a  well-known  continuous- 
time  filter.   It  is  then  possible  to  transform  the  transfer 
function  in  the  s-domaln  into  a  transfer  function  in  the 
z-domaln.   Several  functional  transform  methods  have  been 

used  [5]. 
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a.  Standard  (Impulse  Invariant)  Z-Transform 

The  Standard  Z-Transform  maps  the  transfer  func- 
tion from  the  s-domaln  to  the  z-domaln  by  the  transformation 

sT 
z  =  e   the  value  of  T  and  thus  w  having  previously  been 

decided.   This  method  alv/ays  results  In  the  unit  pulse  re- 
sponse of  the  discrete-time  filter  being  identical  to  the 
samples  of  the  continuous-time  filter  taken  at  Intervals 
T  of  the  Independent  variable,  t,  starting  at  t  =  0 ,  hence 
the  equivalent  name.  Impulse  Invariant  Z-Transform.   There 
is  a  difficulty  in  this  method,  however.   By  Equation  (2.5) 
it  can  be  seen  that  the  frequency  response  of  the  discrete 

time  filter  for  -u  /2  <  w,  <  w  /2  will  involve  the  sum  of 

s   —  d  —  s 

the  values  of  the  frequency  response  of  the  continuous-time 

filter  at  0)  =  to,  +  koj  ,  -<==  <  k  <  ~.   This  effect  is  called 

d     s* 

i  coT  ^ 

aliasing  and  will  result  in  H(e'^   )  not  being  equal  to  H(jco) 

in  the  region  of  interest.   They  may  be  approximately  equal 

if  H(ja))  is  very  small  outside  half  the  sampling  frequency, 

but  no  time  limited  function  (which  an  impulse  response  is) 

can  be  limited  in  fi'equency.   On  the  other  hand,  when  H(jco) 

is  not  small  outside  half  the  sampling  frequency,  H(e'^   ) 

will  bear  no  resemblance  to  the  desired  frequency  response. 

For  this  reason  other  functional  transforms  are  needed. 

b.  The  Bilinear  Z-Transform 

Another  method  of  transforming  a  transfer  function 
from  the  s-domaln  to  the  z-domaln  is  by  the  mapping 
z  =  (1  +  ~)/(l  -  ■—)  which  has  the  Inverse  mapping 
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2  2  —  1 

s  =  ^(^  ^  j^)  .   This  mapping,  however,  distorts  the  frequency 

response.   Therefore  It  Is  necessary  to  counter-warp  the 
desired  radian  frequency  response  before  applying  the  trans- 
formation.  This  still  does  not  yield  an  exact  equivalence 
between  the  two  frequency  responses  because  the  pole-zero 
locations  are  not  matched. 

0.   The  Matched  Z-Transform 

The  Matched  Z-Transform  Is  also  a  Bilinear 
Z-Transforra  using  the  same  bilinear  mapping  as  before.   But 
the  way  the  transfer  function  is  modified  is  to  replace  each 
pole  or  zero  of  H(s)  given  by 


s  -  s.  =  0  (2.8a) 

1 


with  a  pole  or  zero  of  H(z)  given  by 


s.T 
^  (2.8b) 


z  -  z.  =  z  -  e 


This  process  effectively  prewarps  both  the  radian  frequency 
response  and  the  neper  frequency  response. 
d.   Higher  Order  Transforms 

Examination  of  the  effect  of  the  preceding  func- 
tional transform  techniques  reveals  that  the  algorithm  implied 
by  the  transfer  function  in  the  z-dom.ain  is  a  numerical  inte- 
gration routine  v;here,  for  the  Standard  Z-Transform,  the 
routine  is  of  zero-order  (rectangular)  and  for  the  Bilinear 
Z-Transform  it  is  first-order  (trapezoidal).   It  is  possible 
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then  to  hypothesize  using  higher  order  numerical  Integration 
algorithms  for  the  trans form.at Ion  between  the  s-  and  z- 
domalns .   This  would  have  the  effect  of  raising  the  order 
of  the  transfer  function  (which  Is  the  order  of  the  denom- 
inator polynomial)  In  the  z-domaln.   This  Increases  the 
complexity  and  cost  of  the  filter  and  thus  Is  not  usually 
done  explicitly  In  practice,  but  may  be  done  implicitly  as 
observed  in  the  next  section. 

e.   Finite  Impulse  Response  (FIR)  Filters 

When  a  filter's  impulse  response  (which  is  zero 
for  n  <  0)  is  also  zero  for  all  samples  after  some  sample 
h(N) ,  the  filter  is  called  a  finite  duration  impulse  response 
or  simply  finite  impulse  response  (FIR)  filter.   In  this 
case,  the  denominator  polynomial  of  H(z)  is  z"   and  the 

numerator  is   E  h(n)z^~^.   If  an  FIR  filter  is  being  used 

n=0 
to  approximate  a  continuous-time  filter  of  order  less  than 

N,  then  some  combination  of  higher  order  integration  routines 

are  Implicitly  being  performed.   But  the  intent  in  designing 

an  FIR  filter  is  usually  to  approximate  some  property  of 

the  desired  frequency  response  such  as  linear  phase  [6], 

minimum  phase  [7],  or  magnitude  response  [8,9]  often  without 

reference  to  the  pole-zero  locations  of  a  continuous-time 

filter  and  letting  the  order  of  the  filter  be  that  which  is 

needed  to  meet  some  design  criterion  such  as  minim.ax  or 

mean-square  difference. 
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3.   Transfer  Function  Realizations 

Once  the  functional  form  of  the  filter  has  been 
decided  upon.  It  Is  then  necessary  to  design  the  realization 
of  the  algorithm  to  be  performed. 

a.   Definitive  Forms 

The  most  direct  way  to  realize  a  transfer  function 
Is  by  performing  the  operations  implied  by  that  function. 
(This  is  called  the  direct  realization  for  obvious  reasons.) 
The  transfer  function  is  usually  written  as  the  ratio  of 
two  polynomials  in  z,  such  as 


M    ^ 

K(z)  =  ^^^—  (2.9) 

Z  b.z'- 
0=0  ^ 

with  N  _>  M  for  realizabillty . -^  Then  it  is  possible  to 

-N 
multiply  numerator  and  denominator  by  z   to  yield 


Z  a .  z       Z   a   .  z 
Z  b  .z^"      Z  b„  .z"-^ 


It  will  be  shovm  in  Chapter  III,  that  in  some  contexts 
it  is  necessary  that  N  be  greater  than  M  for  realizablllty . 
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Recalling  that  z"  is  the  unit  delay  operator, 
the  product  z   X(z)  Is  the  Z-Transform  of  x(n-k) .   Therefore, 
by  cross  multiplying  the  last  equality  in  (2.10),  taking 
the  inverse  Z-Transform  of  both  sides  term  by  term  and 
letting  b„  be  unity  (without  loss  of  generality),  the  time 
domain  equivalent  of  (2.10)  is  given  by 


N  N 

v(n)  =  ^    a„  .u(n-i)  -  Z  b.,  .v(n-J)       (2.11) 
1=N-M  ^"^         J  =  l  ^^"-^ 


Equation  (2.11)  is  realized  directly  in  Figure  (2-1)  which 
has  2N  unit  delays  indicated  by  z~  .   But  by  defining  an 
intermediate  variable  x(n)  ,  such  that 


N 
x(n)  =  u(n)  -   -  b   .x(n-j)  (2.12) 

j=l   '^ 


it    can  be    shov/n   that    (2.11)    becomes 


N 
v(n)    =      Z        a^   .x(n-i)  (2.13) 

i=N-M  ^^"^ 


Equations  (2.12)  and  (2.13)  represent  the  algorithm  of 
Figure  (2-2)  which  is  known  as  a  canonic  realization.   It 
is  canonic  in  that  it  has  the  least  possible  number  of  delay 
elements  (N) . 

b.   Reduction  to  Lower  Order  Forms 

V/hen  the  numerator  and  denominator  polynomials 
arc  known  in  factored  form,  it  fs  possible  to  realize  the 
transfer  function  as  a  combination  of  lower  order  transfer 

functions . 
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(1)  Parallel  Realization  by  Partial  Fraction 
Expansion.   If  the  transfer  function  has  simple  poles,  then 
it  can  be  written  as  the  sum  of  first  and  second  order  trans- 
fer functions  (sometimes  called  sections)  by  performing  a 
partial  fraction  expansion.   These  can  then  be  realized 
individually  and  then  placed  in  parallel  between  the  input 
and  output  as  in  Figure  2-3.   If  the  transfer  function  has 
multiple  poles,  higher  order  sections  will  be  required  and 
will  generally  result  in  a  higher  number  of  delays  and 
multipliers  than  in  the  canonic  realization. 

(2)  Cascade  Realization  by  Factorization.   In 
the  factored  form  there  is  a  definitive  way  to  realize  the 
overall  transfer  function  by  arbitrarily  associating  zeroes 
and  poles  and  writing  the  transfer  funcT:ion  as  a  product  of 
lower  (usually  first  and  second)  order  sections  as  in  Figure 
2-4. 

(3)  Hybrid  Realization.   The  operations  of  (1) 
and  (2)  above  may  also  be  used  in  many  different  combinations 
to  form  many  different  hybrid  realizations,  such  as  in 
Figures  2-5a  and  2-5b . 

c.   Canonic  First  and  Second  Order  Sections 

Since  any  transfer  function  can  be  realized  as 
a  combination  of  first  and  second  order  sections,  these  low 
order  forms  take  on  a  great  deal  of  importance  in  the  design 
of  discrete  time  filters.   Figures  (2-6a  and  b)  show  the 
canonic  realizations  of  first  and  second  order  filters. 
Jackson  [10]  discussed  variations  of  Figure  (2-6b)  which  are 
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Just  as  efficient  In  terms  of  operations.   Hess  [11] 
extended  these  to  24  forms.   This  work  extends  these  to 
58  forms. 

d.   Other  Forms  of  Realizations 

There  are  several  other  forms  for  the  realization 
of  a  transfer  functions.   Among  these  are  the  so-called  ladder 
structures  [12,13]  found  by  continued  fraction  expansions 
and  the  Fettwels  structures  [1^+]  which  are  found  by  direct 
analogy  to  RLC  ladder  structures.   Both  of  these  types  are 
ladder  structures  so  care  in  terminology  is  necessary  to 
avoid  confusion. 

4 .   The  Discrete  Fourier  Transform 

As  mentioned  in  the  first  paragraph  of  this  section 
the  Discrete  Fourier  Transform  (DFT)  is  a  special  cade  of 
the  Z-Transform.   The  DFT  is  the  Z-Transform  of  a  sequence 
of  length  N  evaluated  at  N  equally  spaced  intervals  around 
the  unit  circle  in  the  z-plane  and  can  thus  be  defined  by 


X(kn)  =  X(z) 


^^■^    ,    >  -jknOT   ^I-"-  f    ^,,nk 
jkOT  =   ^  x(n)e       =   Z  x(n)W 
z=e       n=0  n=0 


0  <  k  <  N-1        (2.1i)) 


where 


n  =  2tt/NT 
W  =  e-J2./N 
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The  Inverse  Discrete  Fourier  Transform  (IDFT)  is  given  by 


,  N-1 
x(n)  =  ^  E  X(kn)W  "^         0  <  n  <  N-1    (2.15) 
^^  k=0  ~   ~       . 


Notice  that  if  n  and  k  are  outside  the  limits  [0,N-1]  then 
(2.14)  and  (2.15)  are  periodic  with  period  N. 

The  IDFT  of  the  product  of  the  DFT's  of  two  sequences 
of  length  N  is  by  definition  reducible  to  the  circular  con- 
volution of  the  two  sequences.   The  process  of  circularly 

2 
convolving  two  sequences  of  length  N  requires  N  multipli- 
cations, v;hile  the  use  of  the  DPT  to  accomplish  the  same 

2 

operation  takes  3N   +  N  multiplications  so  it  is  more  effi- 
cient to  perform  the  convolution  directly.   There  were  many 
early  attempts  to  make  the  DFT  more  efficient,  including 
the  Goertzel  algorithm  [4].   Then  Cooley  and  Tukey  [15] 
found  a  v;ay  to  reduce  the  number  of  operations  in  finding 
the  DFT  which  has  become  knovm  as  the  Fast  Fourier  Transform 
(FFT)  [16-19]. 

There  are  many  variations  of  the  algorithm  now, 
including  algorithms  for  N  a  composite  number  of  mixed  radix, 
algorithms  which  require  sorting  in  time  (decimation  in  time) 
as  in  the  Cooley-Tukey  algorithm  and  algorithms  which  require 
sorting  in  frequency  now  known  generically  as  Sande-Tukey 
algorithms.   Further  reductions  in  the  number  of  operations 
required  has  been  accomplished  by  observing  symmetries  and 
common  factors. 
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The  DFT  has  also  been  widely  used  in  the  discrete 
filter  design  process  [8,  20]. 

5.   Other  Transform  Techniques 

Other  transform  techniques  utilized  In  discrete  time 
signal  processing  Include  the  Chirp  Z-Transform  [21],  Fermat 
Number  Transforms  [22,  23],  and  the  Walsh-Hadamard  Transform 
[21,25]  and  the  Hllbert  Transform  [26]. 

B.   DIGITAL  PROCESSING 

When  the  theory  of  Section  A  Is  applied  to  digital 
processing,  the  discrete  valued  nature  of  the  numerical 
representation  required  produces  nonlinear  effects  which 
disturb  the  solutions  to  what  otherwise  would  be  linear 
equations.   Among  the  problem  areas  of  concern  are  analog 
to  digital  (A/D)  conversion  errors,  coefficient  representa- 
tion (the  approximation  problem)  and  quantization  errors 
Involved  in  arithmetic  operations. 

1.   A/D  Conversion 

The  problem  of  A/D  conversion  is  not  one  which  lends 
Itself  to  solution.  It  is  a  problem  which  must  be  contended 
with,  the  only  way  of  reducing  it  being  to  use  more  signifi- 
cant digits  in  the  digital  signal  representation  thus  making 
this  purely  a  cost  vs.  error  tradeoff  consideration.  Bennett 
[27]  has  presented  spectral  studies  of  A/D  conversion  noise, 
thus  allowing  the  designer  to  represent  this  error  as  an 
injected  noise  at  the  input  of  the  digital  processor.  Often 
the  quantization  level  of  A/D  converters  is  much  larger  than 
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that  of  the  digital  processor  [28].   The  processor  may  then 
include  some  filtering  action  to  reduce  the  effect  of  the 
A/D  conversion  errors. 

2.   The  Approximation  Problem 

When  attempting  to  synthesize  a  digital  filter  as 
an  approximation  of  some  required  characteristic  such  as 
impulse  response  or  frequency  response,  the  determination  of 
filter  coefficients  is  complicated  by  the  finite  word  length 
representation  allowed.   This  causes  error  minimization 
procedures  to  become  iterative  nonlinear  routines. 

The  sensitivity  of  digital  filter  algorithms  to 
shifts  in  the  ideal  coefficient  values  has  been  studied 
extensively.   Kaiser  [29]  concludes  that  for  a  direct  filter 
realization,  the  sensitivity  of  pole  positions  Increases  with 
the  order  of  the  filter.   This  has  been  confirmed  by  Knowles 
and  Olcayto  [30].   Rader  and  Gold  [31],  in  studying  the 
sensitivity  of  second  order  filters,  conclude  that  two 
coupled  first  order  sections  are  less  sensitive  than  a  single 
second  order  section.   But  it  should  be  observed  that  two 
coupled  first  order  sections  require  more  multipliers  than 
canonic  forms  (see  Chapter  III)  and  are  merely  a  general 
state  variable  realization  which  is  not  canonic.   Mantey 
[32]  studied  state  variable  representations  and  concluded 
that  digital  filters  should  be  realized  as  parallel  or  cas- 
cade combinations  of  first  and  second  order  sections.   This 
is  consistent  with  the  conclusions  of  Ka:3ser  above.   Ladder 
structures  in  general  have  been  found  to  be  less  sensitive 
to  coefficient  inaccuracy  [33]. 
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3.   Quantization  Errors 

Arithmetic  operations  in  digital  filters  almost 
invariably  lead  to  the  requirement  to  requantlze  the  result 
of  the  operation  at  some  time  during  one  cycle  time.   Mul- 
tiplication Increases  the  word  length  of  the  signal.   Addi- 
tion and  multiplication  may  exceed  the  dynamic  range  of  the 
numerical  representation.   Floating  point  addition  may  re- 
quire quantization  before  the  operation  since  both  the  sum- 
mand  and  addend  must  have  the  same  exponent. 

The  choice  of  arithmetic  mode  significantly  effects 
the  level  of  quantization  noise.   Since  fixed-point  arith- 
metic is  easier  to  Implement  it  is  most  popular.   But  the 
relative  noise  level  can  be  severe  if  the  full  dynamic  range 
of  the  registers  is  not  being  used  effectively.   This  requii'tib 
scaling  the  driving  function.   Floating  point  arithmetic 
utilizes  the  exponent  for  automatic  scaling.   An  Intermediate 
method  is  known  as  block  floating  point  [B'^l  where  the  expon- 
ent for  a  whole  block  of  data  (such  as  a  single  second  order 
section  in  a  cascade  realization)  is  maintained  in  a  single 
register. 

Oppenheim  and  Welnstein  published  a  review  [351  of 
the  effects  of  quantization  errors  in  digital  filter   and 
Fourier  Transform  algorithms.   This  review  Includes  the  work 
of  Liu  and  Kaneko  [36]  on  floating  point  arithmetic,  Knowles 
and  Edwards  [3?]  and  Gold  and  Rader  [38]  on  fixed  point 
arithmetic,  and  V/einsteln  and  Oppenheim  [39]  where  all  three 
modes  v;ere  compared  on  the  basis  of  signal-to-nolse  ratio. 
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It  was  determined  there  that.  In  general,  floating  point 
arithmetic  was  best  with  block  floating  point  falling 
between  the  other  two  modes . 

The  choice  of  quantization  mode  also  effects  quanti- 
zation.  For  addition,  two's  complement  arithmetic  exhibits 
disastrous  errors  when  overflow  (upper  level  quantization) 
occurs  [^0,  ill].   Zeroing  and  saturation  arithmetic  yields 
better  results  [^2].   For  multiplication  and  addition, 
lower  level  quantization  methods  include  rounding,  trunca- 
tion and  sign-magnitude  truncation.   Jackson  [^3]  has  studied 
the  interaction  of  upper  level  and  lower  level  quantization 
errors . 

Since  quantization  error  is  a  nonlinear  effect, 
experience  has  shown  that  the  phenomenon  of  limit  cycles 
exists  in  digital  systems  just  as  in  continuous  ones.   Limit 
cycles  have  been  extensively  studied  both  in  statistical 
and  deterministic  ways. 

Certain  bounds  on  the  magnitudes  of  limit  cycles 
in  particular  and  accumulated  roundoff  errors  in  general  in 
second  order  sections  have  been  proposed  by  many  authors  and 
are  tabulated  in  Table  2-1. 

These  bounds,  however,  have  been  derived  assuming 
uncorrelated  error  sources,  a  restriction  v;hich  will  not 
be  assumed  in  the  derivations  of  Chapter  IV. 
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Author 


Reference 


Bound 


Condition 


Jackson  (estimate) 

44 

0.5 
1-b 

0<f/f3<| 

Jackson  (bound) 

10 

1 

f^0,fg/2 

1-  a  +b 

Bonzanlgo 

45 

1 

f=0,f/2 

1-  a  +b 

Sandberg  and  Kaiser 

46 

1 

b>0 

4b 
^^1+b 

0<f/f^<| 

(l-b)[l-(a2/4b)] 

1/2 

1 

b<0 

4b 
a>  1+b 

1-  a  +b 

Long  and  Trick 

47 

1 

b<a^/4 

1-  a  +b 

1 

4t 

)^(~-l)<a^<4b 
Vb 

b>0 

(1-^)2 

1+  Vb 

a2<4b2(-^-l) 
-        Vb 

b>0 

(l~b)[l-(a^/4b)] 

1/2 

Yakovri.tz  and  Parker 

2 

1 

b<a^/4 

1-  a|+b 

1+Vb 

b>a^/4 

(l-b)[l-(a^/4b)] 

1/2 

f  =sanpllng  frequency 


1+az  +bz~  =denorninator  of  H(z) 


TABLE  2-1  Present  Bo'jnds  on  Quantization  Error 
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In  statistical  studies  of  quantization  errors,  the 
expressions  for  the  output  error  variance,  assuming  uncor- 
related  errors  and  zero  mean,  uniformly  distributed  statistics, 
usually  Involve  contour  Integrations  of  the  form  [^,^8] 


2 
where  a^      Is  the  output  error  variance 

and  G. (z)  is  the  transfer  function  between  the  1   error 
source  and  the  output.   This  form  of  analysis  requires  the 
determination  of  all  the  G.(z)  and  the  performance  of  all 
the  contour  integrations.   Furthermore,  the  assumption  of 
uncorrelated  error  sources  provides  no  insight  into  the 
effects  of  structure  on  the  output  error  variance.   Nor  does 
this  method  allow  the  effect  of  assumptions  made  about  the 
distribution  and  correlation  of  the  errors  to  be  examined. 
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62(2) 

> 

• 
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y  ' 

H(z)   =  K     +     E     G.(z) 


1=1 


Figure  2-3.     Parallel   Realization  of  H(z) 
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Figure  2-4.  Cascade  Realization  of  H(z) 
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64(2) 

H(z)  =  K^EK^  +  K3G^(z)G2(z)  +  \^^{z)^^{z)^ 


Figure  2-5a.     Hybrid  Realization  cf  H(z) 
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Figure  2-5b.     Hybrid  Realization  of  H(z) 
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Figure  2-6a.  Canonic  First  Order  Section 
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Figure  2-6b.  Canonic  Second  Order  Section 
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III.   CANONIC  REALIZATIONS  OF  SECOND  ORDER  SECTIONS 

A.   INTRODUCTION 

In  section  IIA3b  it  was  shown  that  any  rational  transfer 
function  could  be  expressed  as  a  parallel,  cascade  or  hybrid 
parallel-cascade  combination  of  first-  and  second-order 
transfer  functions  (or  sections).   For  reasons  of  noise 
generation,  coefficient  sensitivity,  economics,  logistics 
and  others,  these  low  order  sections  assume  great  importance 
in  the  field  of  digital  filtering. 

Two  ways  of  realizing  a  second  order  section  (by  the 
direct  or  the  so-called  canonic  realization  (see  section  IIA3c) 
have  already  been  given.   The  "canonic"  form  has  been  and, 
in  general,  still  is  the  way  second  order  sections  are 
realized  in  practice.   However,  when  one  considers  the  second 
order  section  in  its  state  variable  form,  one  realizes  that 
there  are  an  infinite  number  of  solutions  depending  upon 
the  manner  in  which  the  basis  is  represented  in  the  state 
space.   And,  in  fact,  many  of  these  are  canonic  in  the  sense 
of  minimizing  the  number  of  operations  required.   This  mini- 
mization also  implies  a  minimum  number  of  noise  error  sources 
due  to  quantization  after  m.ultiplication  since  the  number  of 
multiplication  operations  is  minimized. 

In  this  chapter,  some  inquiries  are  made  into  the  relation- 
ship between  the  transfer  function  and  the  state  and  output 
equations  of  a  second  order  section.   These  relationships 
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will  then  be  exploited  to  find  a  group  of  state  and  output 
equation  sets  considered  best  In  the  sense  of  minimizing  the 
number  of  quantization  error  sources.   These  sets  are  reduced 
to  a  number  of  "arrays"  for  compactness  of  notation.   Half 
of  the  realizations  represented  by  these  arrays  have  not 
appeared  In  the  literature  to  date  nor  has  such  an  exhaustive 
search  of  allowable  state  equations  been  performed.   The 
next  chapter  will  delve  Into  the  relative  noise  properties 
of  the  realizations  derived  in  this  chapter  with  a  view 
tov;ard  selecting  optimal  structure. 

B.   THE  TRANSFER  FUNCTION 
1 .   Basic  Forms 

The  most  general  form  of  the  second  order  transfer 
function  is  given  by 

-la.      -2 

ur.^  -  ^0    ^1^      ^2^    _  V(z)  (^    ^. 

b„  +  b  z   +  bpZ 

where  v(n)  is  the  output  and  u(n)  the  input. 

In  this  form,  it  appears  that  there  are  six  coeffi- 
cients and  hence  six  sources  of  error  due  to  quantization 
after  multiplication.   It  is  also  possible  to  write  equation 
(3.1)  in  the  form 

-1  x    .-2 
a„  +  a,  z    +  a,  z 

H(z)  =  -0 ^ :4^ (3.2) 

1  +  az   +  bz 


^3 


where 


Cq  "  ^q/'^O  '                    (3.2a) 

"l  "  ^I'^^O  (3.2b) 

a^  =  a^/bp  (3.2c) 

a  =  b^/bQ  (3. 2d) 

b   =  b2/bQ  (3.2e) 


so  that  there  are  only  five  non-unity  coefficients.   (Note 
that  it  is  not  necessary  to  require  the  condition  b^  7^  0 
since  if  b  viere    zero  the  denominator  of  (3.1)  would  not  be 
a  second  order  polynomial  in  z,  hence  H(z)  vjould  not  be  a 
second  order  transfer  function.)   So  H(z)  has  been  written 
in  (3.2)  in  a  form  that  has  at  most  five  quantization  error 
sources . 

From  the  form  (3.2)  it  is  also  possible  to  rewrite 
H(z)  as 


/        -1  ^   -2 

H(z)  =  K  d  +  —^ —^ ^1  (3.3) 

I    1  +  az"-^  +  bz"'^' 


where 


d  =    I  °       =  (3.3a) 

0    a^  =  0  J     I  0   aQ  =  0 


^m 


c  = 


a. 


a. 


a. 


-  a 


a^  ^  0 


«0  =  0 


r 


^  "^0 


^0      ^o'° 


^0  =  ° 


(3.3b) 


a. 


a. 


a. 


a^  7^  0 


a^  =  0 


)  ^  < 


'0    ""O 


^0  ^  ° 


^0  =  ° 


(3.3c) 


K  = 


a 


«0  ^  ° 


«0  =  ° 


0 

1 


^0  ^  ° 


^0  =  0 


(3.3d) 


by  dividing  the  denominator  of  (3.2)  into  the  numerator  in 


ascending  povjers  of  z 
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Another  form  is  possible  if  b  (  or  equlvalently  bp) 
f^   0.   Then  the  transfer  function  may  be  written 


H(z)  =  K' 


d'  + 


c  '  +  e  '  z 


-1      -2 
1  +  az   +  bz 


(3.4) 


where 


a' 


1    ^2  7^  0 


=  0 


1    a^  7^  0 


0    a2  =  0 


(3.4a) 
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, »  = 


a. 


-  1 


a2  7^  0 


a^  =  0 


=  < 


^  V2 


-  1 


\   0 


a.^   ji   Q 


a2  =  0 


(3.^b) 


«  = 


"2 

-  a 

\  =  ^ 

f   ^^2 

^2^0 

^0 

a^   7^   0 

"l 

a_   =    0 

^1 
V  b„ 

ap   =••   0 

a. 


V  t  = 


a^  7^  0 


a^  =  0 


a. 


V  = 


(3.^c) 


ap  7^  0 


a2  =  0 


(3JM) 


by  (iividlng  the  denominator  of  (3.2)  Into  the  numerator  in 
descending  powers  of  z~    .   Note  that  the  condition  b  /  0 
is  equivalent  to  saying  that  there  Is  no  pole  of  H(z)  at 
the  origin  in  the  z-plane .   If  there  ls_  a  pole  at  the  origin, 
then  the  other  pole  is  real  and  two  first-order  sections  are 
recommended  rather  than  one  second-rder  section.   So,  in 
any  practical  second-order  section  it  will  be  considered 
true  that  neither  b„  nor  bp  is  zero. 

It  is  still  true,  in  the  forms  of  (3-3)  and  (3.'^), 
that  H(z)  can  be  realized  with  at  most  five  quantization 
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error  sources.   However,  the  coefficients  K  and  K'  need  not 
be  considered  part  of  the  transfer  function  since  they  are 
scaling  factors.   Most  often,  in  the  practical  realization 
of  higher  order  transfer  functions  by  cascade,  parallel  or 
hybrid  forms  (or  even  in  realizing  a  second  order  system)  a 
scale  constant  will  be  used  which  is  not  K  or  K'  but  rather 
some  number  v;hich,  hopefully,  will  prevent  overflow  in  the 
adders.   Any  gain  factor  necessary  to  restore  the  system  to 
some  overall  gain  factor  is  then  included  at  the  final  out- 
put.  Considering  only  the  terms  within  the  parentheses  of 
equations  (3.3)  and  (3.^)>  then  there  are  at  most  four 
quantization  errors. 

If  it  is  the  case  that  a„  =  0  (implying  an  Infinite 
zero  of  H(z))  or  a^  =  0  (Implying  a  zei-u  at  the  origin), 
then  it  is  possible  to  factor  a  scaling  constant  out  of  the 
rational  part  of  (3.3)  or  (3.^)  respectively,  further  reducing 
the  number  of  error  sources  to  at  most  three  (four  if  the 
scale  factor  is  included) . 

Or  it  could  be  that  one  or  more  of  the  coefficients 
is  an  integer  in  which  case  there  can  be  even  fewer  error 
sources.   But  this  case  is  not  sufficiently  general  to 
warrant  special  consideration  in  developing  algorithms 
for  broad  application  or  mass  production. 

In  the  development  that  follows,  four  cases  will  be 
considered  general  enough  for  Inclusion.   These  are  the  two 
cases  of  equation  (3.3)  with  d  ^  0  or  1  and  the  cases  of 
equation  (3. 'I)  with  d'  =  Q  or  1. 
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It  will  be  seen  that  equation  (3.1)  or  (3.2)  realized 
In  the  "canonic"  way  In  section  IIA3c  Is  Identical  to  one  of 
the  canonic  arrays  to  be  developed,  thus  showing  that  it  is 
not  an  additional  realization. 
2.   Realizability 

There  is  a  subtle  fact  regarding  the  physical  reali- 
zability of  a  digital  filter  which  is  not  readily  apparent 
in  the  mathematical  treatment  of  digital  signal  processing.  ' 
Because  the  operations  involved  in  performing  an  algorithm 
cannot  be  accomplished  instantaneously,  there  must  be  an 
Inherent  delay  between  the  time  that  the  input  arrives  and 
the  time  the  output  is  available  for  further  processing. 
If  the  filter  is  being  used  at  very  low  sampling  rates  com- 
pared to  the  operation  speed  of  the  devices,  this  delay  may 
seem  negligible  but  it  exists  nonetheless.   On  the  other 
hand,  if  one  is  to  design  a  filter  that  is  usable  in  a  broad 
field  of  application  it  is  useful  to  consider  the  highest 
possible  sampling  rate.   Then  the  magnitude  of  this  delay 
is  on  the  order  of  z~  . 

V/hen  the  difference  equation  associated  v^:ith  equation 
(3.2)  is  examined,  the  present  output, 

v(n)  =  aQu(n)  ^•  a,u(n-l)  +  a„u(n-2)  -  av(n-l)  -  bv(n-2), 

(3.5) 
is  seen  to  occur  at  the  same  time  as  the  present  input.   But, 
by  the  previous  discussion,  this  is  impossible  in  a  physical 
sense.   Therefore  we  say  that  equation  (3.2)  is  not,  strictly 
speaking,  realizable. 
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From  (3.5)  It  Is  seen  that  the  factor  which  violates 
the  condition  of  reallzablllty  Is  a  u(n) .   This  Implies  that 
(3.2)  is  not  realizable  because  the  coefficient,  a   In  the 
n\imerator  does  not  have  a  delay  associated  with  It.   Another 
way  of  saying  this  is  that  if  the  numerator  and  denominator 
of  H(z)  are  multiplied  by  the  lowest  power  of  z  which  will 
clear  H(z)  of  all  factors,  z~  ,  then  the  power  of  z  multiplying 
a_  is  equal  to  the  highest  power  of  z  in  the  denominator. 
Therefore,  in  order  to  be  realizable,  any  transfer  function, 
H(z),  written  in  terms  of  positive  powers  of  z,  must  have 
a  numerator  polynomial  of  lower  order  than  the  denominator 
polynomial.   If  H(z)  does  not  meet  this  criterion,  it  is 
always  possible  to  realize  z~  H(z)  where  k  is  an  integer 
large  enough  to  force  the  transfer  function  into  this  condi- 
tion.  In  the  case  of  equation  (3.2),  k  =  1  will  make  the 
transfer  function  realizable  when  a_  ?^  0 .   It  may  be  argued 
that  this  effectively  increases  the  order  of  the  transfer 
function.   This  is  of  little  consequence  since  the  time 
representation  of  the  output  will  be  Identical  except  for 
a  shift  of  k  sampling  intervals. 

On  the  other  hand  it  is  not  alv;ays  necessary  to  make 
a  second  order  section  realizable  in  this  sense.   If  the 
second  order  section  is  part  of  a  cascade  realization  of  a 
higher  order  transfer  function,  for  example,  it  is  only 
necessary  to  ensure  that  the  overall  transfer  function  is 
realizable . 
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The  question  of  realizabillty  will  be  used  In  the 
analysis  of  possible  sets  of  state  equations  in  the  next 
section  In  order  to  reduce  the  number  of  sets  to  be  considered. 


C.   STATE  EQUATIONS 

The  state  equations  for  a  second  order  digital  filter 
consist  of  two  first  order  difference  equations  involving 
the  states  and  input (s)  of  the  filter.   The  output  equatlon(s) 
describes  a  combination  of  the  states  and  input (s)  which  will 
be  seen  external  to  the  filter  as  the  output(s).   Since  a 
transfer  function  is  a  relationship  between  one  input  and 
one  output,  only  state  and  output  equations  with  one  input 
and  one  output  will  be  considered. 

1 .   Basic  Forms 

Define  the  vector,  x(n),  to  be  the  vector  of  states 
in  a  second-order  digital  filter  at  time  n,  so  that 


x(n)  = 


x-|^(n) 
XgCn), 


(3.6) 


Now  define  the  state  and  output  equations  of  the 
filter  as: 


x(n)  =  Ax(n-l)  +  Bu(?) 


(3.7a) 


v(n)  =  Cx(?)  +  6u(?) 


(3.7b) 
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where 

A  Is  a  2x2  matrix 
B  is  a  2x1  matrix 
C  is  a  1x2  matrix 
6  is  a  1x1  matrix  or  scalar. 

The  question  marks  in  parentheses  Indicate  an  uncertainty, 
at  this  point,  of  the  proper  time  index  to  be  used.   The 
n  on  the  left  side  of  equations  (3.7)  has  been  established 
as  a  reference  point  from  which  to  determine  the  other  time 
indices.   Since  it  is  already  known  that  equation  (3.7a)  is 
a  set  of  first  order  difference  equations,  it  follows  that 
A  must  multiply  x(n-l)  thus  establishing  that  time  index. 

Suppose  the  time  index  of  u  in  (3.7a)  is  n-p,  that 
of  X  in  (3.7b)  is  n-q  and  of  u  in  (3.7b),  n-r,  where  p,  q, 
and  r  are  integers  greater  than  or  equal  to  zero  (since 
indices  implying  future  data  ai'e  not  allowed)  .   Then 

x(n)  =  Ax(n-l)  +  Bu(n-p)  (3.8a) 

v(n)  =  Cx(n-q)  +  6u(n-r)  (3.8b) 

Taking  the  Z-transform  of  equation  (3.8a)  and  solving 
for  X(z)  yields 

X(z)  =  [I  -  Az~^j"^  BU(z)  z"P  (3.9) 


where  I  Is  the  Identity  matirx.   Taking  the  Z-transform  of 
equation  (3.8b)  and  substituting  (3.9)  for  X(z)  yields 

V(z)  =  CX(z)z~^  +  6u(z)z"^ 

=  {CCl-Az-^D'^Ez'^P"^^^  +  6z"^}U(z)  (3.10) 

The  factor  inside  the  braces  in  (3.10)  is  thus  a 
transfer  function  between  U(z)  and  V(z),  which  can  be  related 
to  the  transfer  function  in  question.   Two  things  need  to 
be  determined,  (1)  the  values  of  p,  q  and  r  and  (2)  the 
entries  in  the  matrices  A,  B,  C  and  <S  .   The  first  shall  be 
taken  up  here;  the  second  will  be  determined  in  the  next 
section . 

The  quantity  C[I-Az"-'-]~  B  in  (3.10)  is  a  scalar 
formula  in  z~   given  by 


11         4.-1 
$(z)  =  CCl-Az~^]"^B  =  1_L-|^ __        (3.11) 

1  +  az   +  Bz 


where 


Y  =  b^c-^  +  b2C2  (3.11a) 


e  =  a  pC^b^  +  S2l'^2^1  ~  ^^11*^2^2  ~  ^22^1^1     (3.11b) 


a  =  -(a^^  +  a22) 


^  ~  ^11^22  "  ^12^21 


(Note,  b   and  b„  here  are  the  entries  of  the  matrix  B,  not 
the  denominator  coefficients  of  equation  (3.1).)   If  equations 
(3.8)  and  thus  equation  (3.10)  are  to  correspond  to  the  trans- 
fer function  H(z),  then  H(z)  has  to  be  of  the  form. 


H 


(z)  =  6z-^  +  $(z)z-^P"'^^ 


.  -r  ^  (Y-fez-^)z-^P-^q^ 
1  +  az   +  3z 


_  6z-^  +  a6z-^^-^^)  +  B6z-(2+r)  ^  ^,-(P+q)  ^  ,,-(P+q+l) 


1  +  az   +  bz 


(3.12) 


If  the  requirement  Is  now  placed  on  the  numerator 
of  H(z)  to  be  at  most  quadratic  In  nature  (I.e.,  disregarding 
overall  delay,  the  difference  between  the  highest  and  lowest 
power  of  z~      is  no  greater  than  two)  it  is  possible  to  restrict 
the  allov/ed  values  of  p ,  q  and  r. 

First  of  all,  the  only  Indices  available  in  the 
states  of  the  filter  are  n  and  n-1,  so  q  can  only  be  0  or  1. 

Now  the  highest  power  of  z~   in  (3.12)  is  either 
p+q+1  or  r+2  and  the  lowest  Is  either  r  or  p+q .   The  following 
logical  arguments  will  place  further  conditions  on  p  and  r. 
a.   argument  (A) 

If  p+q+1  >_  r+2  and  p+q  ^  r  then  r-q+1  <_  p  _<r-q . 
But  this  Is  a  contradiction,  so  this  case  can  never  arise. 
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b.  argument  B 

If  p+q+1  ^  r+2  and  r  <_  p+q,  then  the  stronger 
condition  is  that  p  >_  r-q+1,  but  the  quadratic  requirement. 
(p+q+1)  -  r  <_  2,  implies  that  p  <_  r-q+1,  so  under  these 
conditions  p  =  r-q+1.   Then  for  q  =  0,  p  =  r+1,  and  for 
q  =  1,  p  =  r. 

c .  argument  C 

If  r+2  >_  p+q+1  and  p+q  <_  r,  then  the  stronger 
condition  is  that  p  ^  r-q,  but  the  quadratic  requirement, 
(r+2)  -  (p+q)  <_  2,  implies  that  p  ^  r-q,  so  that  p  must  be 
such  that  p  -  r-q.   Then  for  q  =  0,  p  =  r,  and  for  q  =  1, 
P  f  r-1. 

d.  argument  D 

If  r+2  _>  p+q+1  and  r  ^   p+q  then  r-q  ^  P  ^  x'-q+l 
and  the  quadratic  requirement,  (r+2)  -  r  <_  2  is  automatically 
satisfied.   Then  for  q  =  0,  p  =  r  or  r+1,  and  for  q  =  1, 
p  =  r  or  r-1. 

Arguments  B  through  D  taken  together  imply  that  p 
and  r  differ  by  at  most  1.   Recalling  that  p  and  r  are 
associated  with  both  indices  of  u  and  only  u  in  equations 
(3.8),  it  can  be  assumed  v.'ithout  loss  of  generality  that 
p  and  r  take  on  only  the  values  0  and  1  since  if  they  are 
greater  than  that,  the  signals  u(n-p)  and  u(n-r)  can  be 
considered  delayed  versions  of  some  other  signal  u' (n)  or 
u'(n-l).   Then  the  signal  u'(n)  or  u'(n-l)  could  be  used 
as  the  input. 
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Table  3-1  lists  all  possible  combinations  of  p,  q 
and  r  such  that  they  take  on  only  the  values  0  and  1,  along 
with  the  associated  transfer  function.   In  addition, the 
values  of  p+q,  p+q+1  and  r+2  are  tabulated  along  with  the 
appropriate  logical  argument  which  applies  to  their  relative 
sizes.   Referring  to  those  arguments  and  the  corresponding 
values  of  p,  q  and  r,  it  is  seen  that  lines  (b)  and  (g) 
of  Table  3-1  violate  the  conclusions  of  the  argument  with 
respect  to  the  relative  values  of  p  and  r.   Thus  the  trans- 
fer functions  of  lines  (b)  and  (g)  cannot  be  related  to  a 
second  order  transfer  function  whose  numerator  is  at  most 
quadratic  in  nature. 

The  state  equations  and  output  equation  of  line(h) 
in  Table  3-1  can  be  seen  to  be  identical  to  those  of  line 
(c)  except  that  the  input  in  both  equations  of  (h)  has  been 
delayed  one  sample  period.   Therefore  the  internal  structure 
would  be  the  same  except  for  a  delay  in  the  input  line. 
The  equations  of  line  (e)  can  also  be  seen  to  be  identical 
in  nature  to  line  (h)  by  rewriting  the  output  equation  of  (e) 
as  v(n-l)  =  Cx(n-l)  +  6u(n-l)  instead  of 

v(n)  =  Cx(n)  +  6u(n),  whereas  in  (h),  v(n)  =  Cx(n-l)  +  6u(n-l) 
In  other  v.'ords ,  the  output  of  the  equations  for  line  (h) 
is  a  delayed  version  of  the  output  corresponding  to  line  (e), 
so  the  structure  v/ill  be  the  same  except  for  a  delay  in  the 
output  line. 

Similarly  the  equations  of  (f)  represent  a  delay 
added  on  the  input  line  of  (a)  and  the  equations  of  (d) 
represent  a  delay  on  the  output  line  of  (a) . 
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These  two  groups  of  essentially  identical  state  and 
output  equations  are  Indicated  as  groups  S  and  T  respectively 
in  Table  3-1.   (Note  that  for  6=0  any  of  the  transfer 
functions  In  Table  3-1  could  belong  to  either  group.  Including 
the  two  that  apparently  violated  the  logical  arguments  since 
for  6=0  the  value  of  r  is  irrelevant.   In  order  to  remain 
as  general  as  possible,  6  will  remain  arbitrary.)   Thus  the 
transfer  functions  associated  with  each  member  within  a 
particular  group  represent  essentially  the  same  transfer 
function  v/ith  the  possible  exception  of  an  overall  delay. 

For  simplicity,  two  transfer  functions  are  chosen  to 
represent  their  respective  group,  namely 


Hg'(z)  =  [6  +  <|.(z)z"'"]z^^  (3.13) 


and 


H^(z)  =  [6'  +  $'(z)]z  ^  (3.1^) 


where  <J>'(z)  has  the  same  form  as  ^{z)  .      Equation  (3.13)  Is 
taken  from  line  (h)  of  Table  3-1  and  (3.1^^)  from  line  (f). 
The  primes  in  (3.1^)  have  been  added  for  purposes  which  will 
become  clear  in  the  next  section.   The  reason  for  the  group 
designations  and  subscripts,  S  and  T,  is  historical  and  will 
be  made  clear  at  the  end  of  this  section.   These  two  forms 
have  been  chosen  because  they  have  an  overall  delay  and  are 
thus  physically  realizable. 
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It  Is  now  possible  to  make  some  specific  connections 
between  the  two  forms  of  H(z)  given  in  equations  (3-3)  and 
(3.^)  and  the  state  equations  associated  with  (3.13)  and 
(3.14)  respectively. 

D.   THE  COEFFICIENTS  OF  THE  STATE  EQUATIONS 

Given  a  transfer  function  in  the  form  of  equation  (3.3) 
(with  an  overall  delay  added  for  realizability)  and  a  set  of 
state  and  output  equations  which  yield  a  transfer  function 
in  the  form  of  (3.13) j  the  two  systems  can  be  made  equal  by 
setting  their  transfer  functions  equal  and  solving  for  the 
appropriate  coefficients.    In  this  case  (disregarding 
scaling  constants) 

cz   ^•  ez  ^   I  -1  -  /;:  X    Yz   +  ez~   i  -1 


H(z)  =  (d  +  —^ '^r-^ -\z^   =  |6  +  LfL__l_J:iL__]z 

1  +  az 


^  z    =   6  +  -^ 3^4 

:   +  bz   y       I    1  +  az   +  3z   / 


=  H3(z)  (3.15) 


Equating  coefficients,  it  is  found  that  for  (3.15)  to  be 
compatible 


a  =  a  =  -(a^^  +  a^^^  (3.l6a) 


The  derivation  which  follows  is  essentially  due  to  Parker 
and  Hess  ['49]  which  fellows  a  similar  effort  by  Jackson  [lO'J. 
The  derivation  is  reiterated  here  for  continuity.   The  only 
difference  is  some  minor , changes  in  notation  and  the  use  of 
realizable  forms  of  the  transfer  function.   Parker  and  Hess 
used  equation  (3.3)  without  an  overall  delay  and  the  state 
equations  corresponding  to  line  (e)  of  Table  3-1  with  a  change 
of  +1  in  all  time  indices  of  the  state  equations  but  no  cnange 

in  the  output  equation. 
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b  =  e  =  a^^a22  -  ^12^21  (3.l6b) 


c  =  Y  =  c^b^  +  c^^  (3.16c) 

d  =  6  (3.l6d) 

^  =  ^  =  ^12^1^2  ""  ^21^2^1  "  ^11^2^2  -  ^22°1^1 

(3.l6e) 

Assuming  that  the  coefficients  a,  b,  c,  d  and  e  are  given, 
it  is  necessary  to  solve  equations  (3.I6)  for  the  entries 
of  the  matrices  of  the  state  and  output  equations  from  which 
(3.13)  was  derived.   Any  combination  of  the  a.  .  j  b.  and  c^. 

XJ       J-  — 

which  yields  the  given  values  of  a,  b,  c  and  e  is  a  valid 
realization.   But  there  are  eight  coefficients  in  the  matrices 
A,  B  and  C  which  im.plies  that  in  some  realizations  there 
could  be  as  many  as  eight  quantization  error  sources.   Further- 
more if  the  values  of  a,  b,  c  and  e  are  given  to  k-bit  accur- 
acy, the  solutions  to  equations  (3. 16)  will  not,  in  general, 
yield  coefficients  to  k-blt  accuracy.   The  approach  taken 
by  Parker  and  Hess  was  to  force  the  solutions  to  be  in  k-blt 
accuracy  by  v;rlting  (3.l6a,  b,  c  and  e)  as 


a  =  -(a,^  +  2i^^  (3.17a) 


^  =  '^ll^'^-22\  -  [^^12^2l\  ^3. 17b) 
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c  =  [b^c^]^^  +  Lb^c^2^  (3.17c) 


(3.l8d) 

where  [  •  ]    indicates  quantization  of  the  product  within. 
Since  there  are  four  equations  in  eight  unknowns,  it  is 
possible  to  choose  an  appropriate  set  of  four  unknowns  arbi- 
trarily and  solve  for  the  other  four.   Furthermore,  the 
solution  is  simplified  by  the  intuitive  realization  that 
quantization  of  product  terms  can  be  avoided  if,  in  the 
products  of  (3.17)5  each  term  except  one  is  unity  or  one  of 
them  is  zero.   Coincidentally ,  it  turns  out  that  this  approach 
also  results  in  only  four  non-zero,  non-unity  coefficients, 
i.e.,  the  four  coefficients  remaining  after  arbitrarily 
choosing  four  to  be  unity  or  zero.   This  approach  thus 
yields  the  mdnimum  number  of  quantization  error  sources  as 
well. 

By  applying  a  set  of  rules  for  the  selection  of 

unity  and  zero  coefficients,  Parker  and  Hess  obtained  eight 

2 
system  matrices   v;hich  v/ould  meet  the  criteria.   Only  two 

of  these  were  unique,  tv/o  others  being  transposes  of  the 


2 

These  system  matrices  were  of  the  form 


A  ;  B 
C  ;  6 


,  I.e., 


partitioning  the  matidces  yields  the 

matrices  A,  B,  C  and  6.   The  phrase  "system 

array"  v/ill  be  preferred  here  for  reasons  to  be  shown  later. 
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first  two,  the  other  four  simply  Involved  the  interchange 
of  rows  or  columns  in  the  first  four. 

The  two  unique  system  arrays  derived  by  Parker  and 
Hess  were 


S   = 
a 


r  -a  -b  1 
10   0 
L  c   e   d  - 


(3.18a) 


-  -d+a) 

-d+a+b) 

1 

1 

1 

0 

c 

c+e 

d 

(3.18b) 


The  S   array  had  been  previously  discussed  by  Jackson.   The 

3. 

transposes  of  these  arrays  are  then 


<- 

-  -a        1 
-b        0 

O       O 

.    1         0 

d    . 

r  -d+a) 

1 

c 

^l' 

-d+a+b) 

1 

c+e 

,       1 

0 

d 

(3.19a) 


(3.19b) 


Jackson  showed  that  the  transpose  of  a  system  array 
yields  a  configuration  identical  to  the  untransposed  config- 
uration except  that  (1)  the  direction  of  signal  flow  is 
reversed  and  (2)  summing  nodes  become  pickoff  nodes  and  vice 
versa.   Jackson  further  showed  that  for  each  unique  system 
array  and  its  transpose,  twelve  configurations  can  be  achieved 
through  flow  graph  manipulations.   The  conclusion  of  Parker 
and  Hess  v/as  then  that  there  v;ere  twenty-four  canonic 
realizations  . 
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The  previous  analysis  v;as  based  on  equating  the 
transfer  function  of  equation  (3-3)  with  the  transfer  function 
of  (3.13).   A  similar  analysis  can  be  carried  out  by  equating 
(3.4)  to  (3.14).   Then 


H(z)  =  (d'  +     "'  -^  f  ""^    ,)z-^  =  (6-  +  _jl:_l_li^__  ) 

1  +  az"  +  bz      '         1  +  az~  +  3z" 


-1    ■   -.   ~1   •   r>       ~ ' 


=  H^(z)  (3.20) 


where  c',  d' ,  e',  6',  y'  and  e'  are  primed  to  distinguish 

them  from  the  analysis  which  precedes,  while  a,  b,  a  and  3 

are  not  primed  since  the  characteristic  polynomial  must  be 

the  same  for  all  realizations.   When  the  coefficients  are 

equated,  hov/ever,  it  is  seen  that  the  equations  are  Identical 

in  form  to  equations  (3.I6).   Thus  following  the  same  set 

of  rules  for  selection  of  unity  and  zero  coefficients,  the 

system,  arrays  which  result  will  have  the  Identical  form  of 

S  and  S   of  equations  (3.18).   The  only  exception  is  that 
a      D 

d  will  be  replaced  by  d',  c  by  c',  and  e  by  e ' ,  which  means 
that  for  the  same  transfer  function  H(z),  the  arrays  which 
result  from  (3.20)  will  differ  from  those  resulting  from 
(3.19)  numerically.   In  order  to  indicate  which  form  of  the 
transfer  function  is  used  to  find  the  entries  in  the  array, 
a  T  will  be  used  for  those  resulting  from  equation  (3.4) 
whereas  an  S  designates  the  ones  found  by  Parker  and  Hess, 
thus  the  rationale  for  calling  the  groups  S  and  T  comes 
from  historical  developm^ent .   The  next  section  v;ill  force 
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another  change  In  notation  (altering  the  subscripts,  e.g. 'a' 
in  S^)  to  avoid  an  historical  conflict  and  the  resulting 
confusion  that  might  be  created. 

In  summary,  the  state  equations  and  associated 
transfer  functions  are  given  by 


x(n) 
v(n) 


=  Ax(n~l)  +  Bu(n-l) 
=  Cx(n-l)  +  6u(n-l) 


-^H(z)  = 


HgCz) 


d  + 


-1  ^ 
cz   +  ez 


-1      -2 
1  +  az   +  bz 


-1 


(3.21) 


and 


x(n) 
v(n) 


=  Ax(n-l)  +  B'u(n-l) 
=  C'x(n)  +  6'u(n-l) 


H(z)  = 


H^(z) 


d'  + 


c  '  +  e  '  z 


-1 


-1 


1  +  az   +  bz 


(3.22) 


E.   CANONIC  ARRAYS 

When  Parker  and  Hess  vjrote  their  state  and  output  equations, 
it  was  done  in  such  a  way  as  to  make  it  possible  to  write 
all  three  equations  as  one  matrix  equation.   Thus 


x(n+l)  =  Ax(n)  +  Bu(n) 


v(n)  =  Cx(n)  +  6u(n) 


(3.23a) 
(3.23b) 
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because 


x(n+l) 
v(n) 


^11 


^21 


'12 


'22 


x(n) 
u(n) 


(3.24) 


Thus,  the  arrays  S  and  S^  of  equations  (3. 18)  could  truly 
be  called  matrices.   Examination  of  the  entries  In  Table  3-1 
shows  that  by  raising  the  time  indices  of  the  state  equations 
of  line  (e)  by  1  to  coincide  with  (3.23a),  the  same  result 
is  achieved.   It  is  also  possible  to  write  the  state  and 
output  equations  of  lines  (c)  or  (h)  directly  as  one  matrix 
equation.   These  three  lines  are  all  in  group  S.   When  one 
tries  to  do  the  same  thing  with  the  equations  in  group  T 
it  is  found  to  be  Impossible.   Therefore,  the  arrays  asso- 
ciated with  group  T  cannot  be  called  matrices.   Hence,  in 
this  work  only  the  description  "array"  will  be  used  when 
referring  to  the  3x3  arrays  S  or  T .   The  transpose  of  an  array 
is  defined  as  though  it  were  a  matrix.   A,  B,  C  and  6  continue 
to  play  the  role  of  matrices  and  will  be  referred  to  as  such. 

As  m.entioned  earlier,  Jackson  showed  that  there  were 
six  realizations  possible  for  an  array  and  six  for  its  trans- 
pose.  This  can  be  accom.pllshed  by  flow  diagram,  manipulations 
or,  equlvalently ,  by  m.anlpulatlons  of  the  equations  Involved. 
Tv7o  of  the  six  are  Identical  except  that  in  one  d  =  0  and 
in  the  other,  d  =  1.   In  the  remaining  four  it  v,'as  claimed 
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that  d  had  to  be  unity.   In  fact.  It  will  be  shown  that  there 
are  nine  manipulations  of  the  equations  possible  for  the 
state  equations  of  group  S  and  that  each  allows  d  to  be 
zero  or  unity.   There  is  a  realization  for  all  36  of  these 
cases  (the  nine  manipulations  of  the  array  S  ,  nine  of  S  , 
each  with  d  =  0  and  d  =  1) .   Figures  3-1  through  3-l8  show 
these  realizations.   In  addition,  there  are  the  36  transpose 
configurations  of  these.   To  allow  the  realization  of  any 
transfer  function  with  a  single  realization,  composite 
realizations  are  included  which  usually  require  an  additional 
summer  and  which  change  the  configuration  from  d  =  0  to  d  =  1 
by  the  use  of  a  single  switch. 

Furthermore,  the  state  equations  in  group  T  can  also 
be  m.anipulated  to  yield  different  l].ow  diagrams.   There  is 
not  as  much  flexibility  in  this  group,  however,  and  only  forty- 
four   unique    configurations  have  been  found  (compared  to 
72  for  group  S).   Figures  3-19  through  3-22  show  the 
untransposed  realizations  for  group  T. 

Since  previous  work  has  referred  to  the  realizations 
in  a  manner  which  is  not  adaptable  to  a  one-to-one  corres- 
pondence V7ith  the  realizations  derived  here,  the  notation 
needs  to  be  changed  to  avoid  confusion. 


A  switch  v;ill  be  indicated  as  follows  (see  Figure  3-23)  ■ 

a)  A  switch  which  is  normally  closed  for  d  =  1  and  open 
for  d  =  0  is  represented  as  a  multiplication  by  d. 

b)  A  switch  which  is  normally  open  for  d  =  1  and  closed 
for  d  =  0  is  shown  as  a  multiplication  by  (1-d). 
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Call  an  array  In  the  form  of  equation  (3.l8a)  an 

"M"  array;  an  array  in  the  form  of  equation  (3.l8b)  is  an 

"N"  array.   When  the  array  is  realized  for  the  form  of  the 

transfer  function  and  state  equations  of  (3.21),  precede 

its  designation  by  an  "S" .   In  this  case  the  entries  of  the 

array  include  the  unprimed  quantities:  c,  d  and  e.   Table 

3-2a  lists  all  the  SM  array  manipulations.   Table  3-2b  lists 

those  of  the  SN  array.   SM^_  compares  to  the  S   array  of 

u  u  a 

h 

(3.18a).   SN   compares  to  S  in  (3.l8b). 

When  the  array  is  realized  for  the  form  of  the  trans- 
fer function  and  state  equations  of  (3.22),  precede  its 
designation  by  a  "T" .   In  this  case  the  entries  of  the  array 
include  the  primed  quantities:  c',  d'  and  e'.   Table  3-3a 
lists  all  the  TM  array  manipulations.   Table  3-3b  shows  the 
single  array,  TN  .   This  compares  to  the  S   array  with 
primed  quantities  replacing  the  unprimed.   Similarly  TM 
compares  to  S   with  primed  entries. 

For  group  T  it  vrill  be  shown  that  the  transpose 
configurations  are  not  the  transposes  of  Tables  (3-3)  but 
rather  correspond  to  those  of  Tables  3-2  by  substituting 
primed  quantities  where  applicable. 


h 

The  subscripting  notation  will  be  explained  after  an 

examination  of  the  process  of  manipulating  the  state 
equations . 
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The  development  of  these  realizations  follows.   (Only 
operations  which  do  not  increase  the  number  of  non-zero, 
non-unity  multiplications  are  allowed.) 

Figure  3-la  shows  the  flow  diagram  of  the  array, 
SMqq  with  d  =  1.   The  state  and  output  equations  for  this 
realization  are,  according  to  equation  (3.21): 

x^(n)  =  -ax^(n-l)  -  bx2(n-l)  +  u(n-l)        (3.25a) 

X2(n)  =  x^(n-l)  (3.25b) 

v(n)  =  cx^(n-l)  +  ex2(n-l)  +  du(n-l)         (3.25c) 

Figure  3-lb  shows  the  realization  of  these  equations 

for  d  =  0.   This  configuration  corresponds  to  the  realization, 

S   ,  of  previous  work.   Figure  3-la  corresponds  to  S  „. 
ax  Sid 

Figure  3-lc  exhibits  the  composite  realization  which 
allows  d  to  be  either  0  or  1 .   When  Parker  and  Hess  said  there 
were  2^    canonic  realizations  of  a  transfer  function,  the 
truth  was,  that  they  found  four  realizations  for  a  transfer 
function  with  d  =  0  and  twenty  for  a  transfer  function  with 
d  =  1.   These  are  never  the  same  transfer  function.   There 
is  a  class  of  functions  with  d  =  0  and  a  mutually  exclusive 
class  with  d  =  1.   Correspondingly,  there  is  a  class  of  func- 
tions with  d'  =  0  and  a  mutually  exclusive  class  with  d'  =  1. 
Taken  together,  there  can  be  overlap  between  the  classes  of 
d  and  d' .   That  is  if  d  =  0,  d'  could  be  0  or  1,  etc.   In 


70 


total  there  are  four  classes  given  by  (d  =  0,  d'  =0), 
(d  =  0,  q'  =  l),(d  =  1,  d'  =  0)  and  (d  =  1,  d'  =  1)  which 
include  all  second  order  transfer  functions.   The  inclusion 
of  a  composite  realization  is  in  the  interest  of  logistics. 
Each  composite  realization  can  be  used  for  any  second  order 
transfer  function. 

Proceeding  with  the  development  of  the  other  forms, 
consider  equation  (3.25c).   This  equation  can  be  rewritten 
by  adding  and  subtracting  the  quantity  bxp(n-l)  to  yield  a 
nev/  output  equation 


v(n)  =  cx^(n-l)  +  (b+e)x2(n-l)  +  du(n-l)  -  bX2(n-l) 

(3.26) 

The  quantity  -bXp(n-l)  is  one  which  is  formed  in  calculating 
x^ (n)  so  it  is  available  to  be  used  in  (3.26)  without  adding 
another  multiplier  to  the  realization. 

Equation  (3.26)  together  with  (3.25a  and  b)  are 
realized  in  Figure  3-2a  for  d  =  1.   In  Figure  3-2b,  d  =  0 
and  3-2c  and  d  shov;  the  composite  realization.   These  are 
realizations  of  SM„,  v/here  the  array  is 


-a  -b  1 
10  0 
c    (b+e)-b   d 
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On  the  other  hand  it  Is  possible  to  rewrite  equation 
(3.25a)  by  adding  and  subtracting  ex2(n-l)  to  the  right  hand 
side.   Then 


x-,^(n-l)  =  -ax^(n-l)  -  (b+e)x2(n-l)  +  u(n-l)  +  ex^Cn-l) 

(3.27) 
Equation  3.27  together  with  equations  (3.25b  and  c), 
the  original  second  state  equation  and  the  original  output 
equation,  are  realized  in  Figures  3-3a,  b,  c  and  d  with 
d  =  1,  d  =  0  and  two  composites,  respectively.   In  this  case 
the  array  is  SM   given  by 


-a  e-(b+e)  1 
10  0 
c      e       d 


It  is  now  possible  to  describe  the  subscript  notation  used. 
In  SM  _  there  are  no  manipulations  of  the  original  array. 
In  SM   and  SM   the  second  subscript  is  changed  to  indicate 
a  change  in  the  second  column  of  the  array.   When  the  sub- 
script is  1  it  indicates  adding  and  subtracting  the  coeffi- 
cient of  the  first  row  in  that  column  to  the  coefficient 
of  the  last  rov;  in  that  column.   A  subscript  2  indicates 
the  opposite  operation.   These  operations  can  be  performed 
in  the  first  column  as  well,  or  in  both  columns  at  the  same 
time  and  in  the  same  or  opposite  directions.   A  complete  set 
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of  arrays  with  all  these  operations  are  given  in  Tables  3-2a 
and  b.   Only  the  untransposed  arrays  for  group  S  are  given 
in  the  tables.   Figures  3-1  through  3-l8  show  all  the  untrans- 
posed configurations  implied  by  Table  3-2,  whereas  Figures 
3-2^  through  3-^1  show  the  realizations  found  from  the 
transposes  of  the  arrays  of  Table  3-2. 

Figure  3-5a  is  the  form  historically  called  the 
"canonic"  realization  if  a„  is  factored  out  of  the  numerator 
of  equation  (3.2) . 

For  the  arrays  of  group  TM,  manipulations  are  only 
possible  between  the  two  entries  corresponding  to  a^,  and 
Cp,  that  is  the  first  entry  in  column  1  of  the  array  and  the 
last  entry  in  column  2.   (All  manipulations  in  group  S  were 
accomplished  within  the  same  column.)   There  are  appai-ently 
no  ways  to  rearrange  the  equations  of  array  TN  without 
introducing  more  multipliers  and  summers. 

To  change  TM„  into  TM  ,  consider  the  state  and  output 
equations  for  TM  .   From  equation  (3.22), 


x^(n)  =  -ax^(n-l)  -  bx2(n-l)  +  u(n-l)  (3.28a) 


X2(n)  =  x^(n-l)  (3.28b) 


v(n)  =  c'x^(n)  +  e'x2(n)  +  du(n-l)  (3.28c) 

By  adding  and  subtracting  the  quantity  ax2(n)  on  the 
right  side  of  (3.28c)  the  output  equation  becomes 
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v(n)  =  c'x^(n)  +  (a+e')x2(n)  -  ax2(n)  +  du(n-l)    (3.29) 


But  by  equation  (3.28b),  xAn)    =  x,(n-l)  so 


v(n)  =  c'x^(a)  +  (a+e')x2(n)  -  ax^(n-l)  +  du(n-l)  (3.30) 


The  quantity  -ax, (n-1)  is  one  which  is  formed  in 
calculating  x-,(n)  so  that  it  is  available  to  be  used  in  (3.30) 
without  adding  another  niultlpller  to  the  realization. 

Equation  (3.30)  along  with  (3.28a  and  b)  are  realized 
in  Figure  3-20  as  TM  .   The  opposite  manipulation  indicated 
by  TMp  in  Table  3-3  is  shown  in  Figure  3-21. 

The  un transposed  realization  of  TN   is  given  in 

Figure  3-22. 

5 
TM,  and  TMp  have  no  transpose  configurations,   but 

TM^  does.   It  is  not  true,  however,  that  the  transpose  of  TMq 
is  realized  by  reversing  the  flow  in  Figure  3-19  and  inter- 
changing summing  nodes  and  branching  nodes.   The  realization 
is  found  by  applying  the  state  equations  of  equation  (3.22). 
Furthermore,  taking  SM. .  ,  replacing  c,  e  and  d  by  c',  e'  and 
d'  respectiveli*  and  using  the  state  equations  of  (3.22)  yields 
another  set  of  realizations  for  H(z).   These  realizations  are 
designated  TM . .  (with  i  and  j  corresponding  to  the  subscripts 

-'-J 

of  SM*).   Note  that  TM^  and  TM^„  are  the  same  realization, 
ij  0       00 


The  same  correspondence  is  true  for  the  transpose  of 
J 


TNq.   Taking  SN^.,  replacing  c,  e,  and  d  by  c',  e'  and  d' 


5lf  the  TMi  and  TM2  arrays  are   transposed  it  is  seen  that 
five  multipliers  would  be  required,  thus  they  would  no  longer 
be  canonic  in  terms  of  multipliers  and  error  sources. 
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respectively  and  using  the  state  equations  of  3.22  yields 
the  realization  TN^ . ,  with  TN^^  the  same  as  TNq  . 

Figures  3-^2  through  3-59  show  all  the  transpose 
configurations  for  the  T  forms. 


F.   CONCLUSIONS 

This  development  has  included  a  very  exhaustive  search 
of  allowable  state  equations  and  transfer  functions  for 
realizations  of  second  order  digital  filters,  canonic  in 
the  number  of  delay  elements  and  multipliers  (and  thus  quan- 
tization error  sources  after  multiplication)  and  also  canonic 
in  that  the  multiplier  coefficients  are  linear  combinations 
of  the  coefficients  once  they  are  given  in  the  S  or  T  form 
of  the  transfer  function. 

In  brief,  the  arrays  of  Tables  3-2  and  3-3  V7hen  combined 
with  the  state  equations  of  (3.21)  and  (3.22),  respectively, 
yield  the  realizations  of  Figures  (3-1)  through  (3-22).   The 
transposes  of  the  arrays  of  Tables  3-2  when  combined  with 
both  sets  of  state  equations  yield  the  realizations  of  Figures 
(3-2^)  through  (3-^1)  for  Equations  (3.21)  and  Figures  (3-^2) 
through  (3-59)  for  Equations  (3.22). 

In  total,  these  realizations  consist  of  36  realizations 
for  transfer  functions  with  d  =  1  and  3^  for  d  =  0,  as  well 
as  22  realizations  for  d'  =  1  and  22  for  d'  =  0.   That  is, 
for  a  particular  transfer  function,  there  are  58  realizations 
since  d  cannot  be  both  0  and  1  nor  can  d'  be  both  0  and  1. 
Included  are  composite  realizations  which  allow  d  or  d'  to 
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be  either  0  or  1.   These  require  the  inclusion  of  a  switch 
to  indicate  the  particular  value  and  sometimes  an  additional 
summing  device. 

The  concept  of  "canonic  arrays"  introduced  here  made 
this  development  possible.   If  one  forces  the  array  to  play 
the  role  of  a  matrix  then  only  the  state  equations  of  the 
S  forms  are  possible.   This  forces  the  transfer  function 
also  to  be  in  the  S  form.   The  T  forms  only  arise  when  one 
allows  for  the  concept  of  canonic  array  or,  equivalently , 
separating  the  operations  of  the  state  equations  from  those 
of  the  output  equations.   This  allows  the  time  index  of  the 
states  on  the  right  hand  side  of  the  equations  to  be  different. 
This  factor  is  essential  in  developing  the  T  forms. 
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Figure  3-1  a.     Realization  of  SMqq  with  d= 
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Figure  3-1 b.     Realization  of  SM^q  with  d=0 
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Figure  3-1  c.  Composite  Realization  of  SVlr^n,  with  d=0  or  1 
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Figure  3-2a.  Realization  of  SMq^  with  d=l 


Figure  3-2b.  Realization  of  SMq^  with  d=0 
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Figure  3-2c.     CoiMposite  Realization  of  SMq-.  with  d=0  or  1 
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Figure  3-2d.  Composite  Realization  of  SMq-j  with  d=0  or  1 
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Figure  3- 3a.  Realization  of  SMq2  with  d=l 
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Figure  3- 3b. 


Realization  of  SMq^  with  d=0 
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Figure  5- 3c.  Composite  Realization  of  S'A^^   with  d=0  or  1 
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Figure  3-3d.   Composite  Realization  of  SMq^  with  cl==0  or  1 
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Figure  3-4a.  Realization  of  SM,^  with  d=l 
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Figure  3-4b.     Realization  of  SM-jq  v/ith  d=0 
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Figure  3-4c.  Composite  Realization  of  SM,q  with  d=0  or  1 
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Figure  3-4cl.     Composite  Realization  of  SM,q  wtih  d=0  or  1 
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Figure  3-5a.     Realization  of  SM,,  with  d=l 
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Figure  3-5b.  Realization  of  SM,-,  with  d=0 
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Figure  3-5c.     Composite  Realization  of  SM^,  with  d=0  or  1 
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Figure  3-5d.     Composite  Realization  of  SM,,  wtih  d=^0  or  1 
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Figure  3-6a.     Realization  of  SM,^  with  d=l 
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Figure  3-6b, 


Realization  of  Sl^.n  with  d=0 
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Figure  3-6c.  Composite  Realization  of  SM,^  with  d=0  or  1 
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Figure  3-6d.  Composite  Realization  of  SM,^  wtih  d=0  or  1 
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Figure  3-7a,     Realization  of  SM^^,  wtih  d=l 
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Figure  3-7b.     Realization  of  SM^q  with  d=0 
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Figure  3- 7c.  Composite  Realization  of  SM^q  with  d=0  or  1 


Figure  3-7d.  Composite  Realization  of  SM2Q  with  d=0  or  1 
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Figure  3-8a.  Realization  of  SM^i  with  d=l 
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Figure  3-8b.     Realization  of  SMp,  with  d=0 
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Figure  3-8c.  Composite  Realization  of  SMp,  with  cl=0  or  1 


Figure  3-8d.  Composite  Realization  of  SM^-j  with  d=0  or  1 
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Figure  3-9a.  Realization  of  SM^p  with  cl=l 
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Figure  3-9b.  Realization  of  SM22  with  d=0 
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Figure  3-9c.  Composite  Realization  of  SM,,  with  d=0  or  1 
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Figure  3-9d.  Composite  Realization  of  SH22  with  d=0  or  1 
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Figure  3-lOa.  Realization  of  SNqq  with  d=l 
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Figure  3-lOb.     Realization  of  SN^^  with  d=0 
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Figure  3-lOc.  Composite  realization  of  SN^q  with  d=0  or  1 


96 


u(n-l) 


Figure  3-11  a.     Realization  of  SNq,  with  d=l 


Figure  3-11 b.     Realization  of  SN, .   with  d=0 
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Figure  S-llc.  Composite  Realization  of  SNq.,  with  d=0  or  1 
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Figiire  3-11d.     Composite  Realization  of  SN'q-]  wtih  d=0  or  1 
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Figure  3-1 2a.  Realization  of  SNqj,  with  d=l 


Figure  3-12b.  Realization  of  SN^^^  with  d=0 
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Figure  3- 12c.  Composite  Realization  of  SN^^,  with  cl=0  or  1 


Figure  3-1 2d.  Composite  Realization  of  SfL^  with  d=0  or  1 
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Figure  3-1 3a.     Realization  of  SN^g  with  d=l 
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Figure  3-1 3b, 


Realization  of  SN.q  v/ith  d=0 
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Figure  3-1 3c.     Composite  Realization  of  SN^^  with  cl=C  or  1 


Figure  3-13d.     Composite  Realization  of  SN^q  with  d=0  or  1 
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Figure  3-1 4a.  Realization  of  SN^^  with  d=l 


Figure  3-14b.  Realization  of  SN^^  with  d=0 


103 


(Try 


Figure  3-14c.     Comno<;ite  Realization  of  SN,,  with  d=0  or  1 
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Figure  3-14d.  Conposite  Realization  of  SN^^  with  d=0  or  1 
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Figure  3-15a.     Realization  of  SN-.^  with  d=l 
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Figure  3- 15b.  Realization  of  SN12  with  d=0 
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Figure  3-15c.     Composite  Realization  of  SH.o  with  d=0  or  1 


Figure  3-15d.     Composite  Realization  of  SN^2  ^^"^^  ^""^  ^"^  ^ 
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Figure  3-16a.  Realization  of  SN^q  with  d=l 


Figure  3-1 6b.  Realization  of  SN^q  with  d=0 
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Figure  3-16c.     Composite  Realization  of  SN2Q  with  d=0  or  1 
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Figure  3-16d.     Composite  Realization  of  SN2Q  with  d=0  or  1 
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Figure  3-1 7a.     Realization  of  SN^-j  with  d=l 
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Figure  3-1 7b.     Realization  of  SN^i  with  d=0 
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Figure  3-1 7c.  Composite  Realization  of  SN^,  with  d=0  or  1 
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Figure  3-1 7d.  Composite  Realization  of  SN^-j  with  d=0  or  1 
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Figure  3-18a.  Realization  of  SN^o  with  d=l 
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Figure  3-T8b.  Realization  of  SN22  with  cl=0 
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Figure  3-"!8c.  Composite  Realization  of  SN^,^  with  d=0  or  1 
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Figure  3-18d.     Composite  Realization  of  SN22  with  d--0  or  1 
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Figure  3-19a.  Realization  of  TMq  with  d'=l 
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Figure  3-"!9b.     Realization  of  TMq  with  cl'--^0 
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Figure  3-1 9c.  Composite  Realization  of  TMq  wtih  d'=0  or  1 
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Figure  3-20a.  Realization  of  TM-i  with  d'=l 
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Figure  3-20b.  Realization  of  TM^  with  d'=0 
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Figure  3-20c.     Composite  Realization  of  TM-,  with  d'=0  or  1 
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Figure  3-20d.     Composite  Realization  of  TM-j  with  d'=0  or  1 
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Figure  3-21  a.     Realization  of  TMp  with  d'=l 


Figure  3-21b.     Realization  of  TM^  with  d'=0 
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Figure  3-21 c.  Composite  Realization  of  TMp  with  d'=0  or  1 
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Figure  3-21  d.  Composite  Realization  of  TM2  with  d'=0  or  1 
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Figure  3-22a.     Realization  of  TNq  with  d'=l 


Figure  3-22b.     Realization  of  TNq  with  d'=0 
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Figure  3-22c.  Composite  Realization  of  TNq  with  d'=0  or  1 
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Figure  3-23.  Equivalent  symbols  for  digital  switches 
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Figure  3-24.  Realization  of  SM^J  with  d=0  or  1 


Figure  3-25. 


Realization  of  SMq,  with  d^O  or  1 
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Figure  3-26.     Realization  of  SMq2  with  d=0  or  1 
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Fiugre  3-27.  Realization  of  SM  *  with  d=0  or  1 
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Figure  3-28.  Realization  of  SM,*  with  d=0  or  1 
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Figure  3-29.     Realization  of  SU.^  with  d-0  or  1 
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Figure  3-30.  Realization  of  SM2J  with  d=0  or  1 


Figure  3-31.  Realization  of  SM2^  with  d=0  or  1 
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Figure  3-32.     Realization  of  SM^*  with  d=0  or  1 
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Figure  3-33.  Realization  of  SNqJ  with  d=0  or  1 


Figure  3-34.  Realization  of  SNq|  with  d=0  or  1 
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Figure  3-35.  Realization  of  SNq^  with  d=0  or  1 


Figure  3-36.  Realization  of  SN^g  with  d=0  or  1 
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Figure  3-37. 


Realization  of  SN,j  with  d=0  or  1 


Figure  3-38.  Realization  of  SN^2  with  d=0  or  1 
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Figure  3-39.  Realization  of  SHr^l  with  d=0  or  1 
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Figure  3-40.     Realization  of  SN^]"  with  d=0  or  1 
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Figure  3-41.  Realization  of  SN22  with  d=0  or  1 
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Figure  3-42.  Realization  of  TMq*  with  d'=0  or  1 


Figure  3-43.  Realization  of  TM^^  with  cl'=0  or  1 
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Figure  3-44.  Realization  of  TM^^  with  d'=0  or  1 
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Figure  3-45.  Realization  of  TM^J  with  d'=0  or  1 
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Figure  3-46.  Realization  of  TM,|  with  d'=0  or  1 
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Figure  3-47.  Realization  of  1V<,^  with  d'=0  or  1 
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Figure  3-48.     Realization  of  ll^^l  with  cl'=0  or  1 


Figure  3-49.     Realization  of  TM^j  with  d'=0  or  1 
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Figure  3-50.     Realization  of  TM22  with  d'=0  or  1 
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Figure  3-51.     Realization  of  TN^q  with  d'=0  or  1 


Figure  3-52.     Realization  of  TN^j^  with  d'=0  or  1 
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Figure  3-54. 


Realization  of  TN^J  with  d'=0  or  1 
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Figure  3-55.  Realization  of  TN  *  with  d'=0  or  1 


V -_y     5'    vfn] 


Figure  3-56.     Realization  of  TN^J  with  d'-O  or  1 
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Figure  3-57.     Realization  of  TN^J  v/ith  0=0  or  1 
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Figure  3-58.     Realization  of  llij:  with  d'=0  or  1 
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IV.   ERROR  ANALYSIS  OF  RECURSIVE  DIGITAL  FILTERS 

A.   INTRODUCTION 

The  effect  of  finite  precision  arithmetic  on  the  perfor- 
mance of  digital  filters  has  been  considered  from  the  deter- 
ministic (limit  cycle)  and  probabilistic  (random  noise)  point 
of  view  by  several  authors  [3,Ji4_i|8].   Calculation  of  output 
noise  due  to  quantization,  such  as  round-off  after  multipli- 
cation of  signals  by  constant  factors,  usually  involves  a 
difficult  contour  integration  [i|8]  which  does  not  readily  lend 
itself  to  physical  interpretation  in  terms  of  filter  structure 
or  configuration  for  purposes  of  comparison.   In  addition, 
the  usual  noise  formulas  are  based  upon  the  assumption  that 
the  quantization  error  sources  within  the  filter  are  uncor- 
related.   Although  this  assumption  has  been  confirmed  as 
reasonable  by  several  authors  [35,50], it  is  interesting  to  note 
that  under  the  conditions  of  a  limit  cycle,  quantization 
errors  within  the  filter  are  highly  correlated.   Thus  the 
influence  of  correlation  on  noise  calculations  remains  a 
question. 

In  this  chapter,  several  results  are  presented.   First, 
following  a  procedure  introduced  by  Parker  and  Yakowltz  [3], 
but  without  assuming  uncorrelated  error  sources,  the  covariance 
matrix  of  the  state  errors  and  the  variance  of  the  output 
error  are  expressed  in  terms  of  the  correlation  matrices  of 
the  quantization  error  sources.   This  enables  the  effects  of 


iThis  approach  was  presented  at  the  Arden  House  Workshop 
on  Digital  Signal  Processing  [57]. 
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correlation  between  sources  to  be  taken  into  account.   The 
results  are  presented  in  a  general  form  which  enables  the 
structure  of  the  filter  to  be  introduced  independently. 
More  specifically,  the  expression  for  the  output  error 
variance  is  applied  to  second  order  sections  and  then  to  the 
canonic  realizations  of  Chapter  III. 

Experimental  verification  of  the  results  are  presented 
in  Appendix  D  . 

B.   STATE  SPACE  FORMULATION 

In  Chapter  III  the  equations  for  a  digital  filter  in 
state  variable  form  are  given  by 

x(n)  =  Ax(n-l)  +  Bu(n-l)  (^.la) 

v(n)  =  Cx(n-l)  +  6u(n-l)  C^.lb) 


2 

for  the  S  forms  (see  Chapter  III,  Equations  (3.21))  ,  and 


x(n)  =  Ax(n-l)  +  B'u(n~l)  {^ .2a) 


v(n)  =  C'x(n)  +  6'u(n-l)  (4.2b) 


^-he  use  of  the  terminology  S  forms  (or  T  forms)  is  not 
restricted  to  second  order  filters.   The  state  and  output 
equations  in  matrix  form  are  in  general  applicable  to  n^"  - 
order  filters.   Even  to  the  transfer  function  of  an  n^"  order 
filter,  the  S,T  terminology  applies  in  general.   S  means 
dividing  the  numerator  by  the  denominator  in  ascending  powers 
of  z-1;  T  means  to  divide  in  descending  order. 


l^^B 


for  the  T  forms  (from  Equations  (3.22)),  where  A,  B,  B',  C, 
C,  6  and  6'  are  coefficient  matrices;  x(n),  the  vector  of 
state  variable  at  time  nT;  u(n),  the  input;  and  v(n)  the 
output . 

If  one  assumes  finite  precision  arithmetic  in  carrying 
out  the  digital  filter  algorithm,  then  Equations  (4.1)  become 


x*(n)  =  [Ax*(n-1)]   +  [Bu(n-l)]  (4.3a) 


v*(n)  =  [ex*(n-l)]   +  [6u(n-l)]  (4.3b) 


where  the  asterisk  denotes  rounded  signal  variables  and 

[   ]   denotes  the  process  of  quantization  after  multiplication 


of  a  previously  quantized  signal  variable  by  a  multiplier 


*  th 

That  is,  if  x.(n)  is  the  i    component  of  x*(n)  and  a  .  is 


the  ij    entry  in  A,  then  the  i   component  of  [Ax*(n-1)] 

P      *  ^ 

is  given  by   E  [a..x.(n-l)]   where  p  is  the  order  of  the 

3    '^~ 
filter. 

Following  Yakowitz  and  Parker  [  2  ],  the  following  bounded 
vectors  can  be  defined: 


e(n)  =  Ax«(n-1)  -  [Ax*(n-l)]q  +  Bu(n-l)  -  [Bu(n-l)]q 


(4.4a) 


^Algorithms  which  quantize  after  summation  are  also 
possible  and  have  been   studied  [   ].   Further  discussion  of 
this  and  other  methods  are  considered  in  Section  D. 
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c(n)  =  Cx*(n-1)  -  [Cx*(n-1)]   +  6u(n-l)  -  [6u(n-l)] 
-  -       q  q 

(4.ilb) 

These  are  the  quantization  errors  Introduced  Into  the  filter 
at  each  time,  nT.   There  can  be  several  contributions  to  each 
component  of  e(n)  depending  upon  the  number  of  multipliers 
In  the  appropriate  row  of  A  and  B  which  require  the  product 
to  be  quantized;  and  similarly  for  G(n). 

If  one  defines  the  error,  y(n),  as  the  difference  between 
the  states  of  the  finite  precision  realization  and  those  of 
the  ideal  infinite  precision  realization,  then 

y(n)  =  x{n)    -   x*(n) 

=  Ax(n-l)  +  Bu(n-l)  -  [Ax»(n-l)]q  -  [BuCn-l)]^ 

(4.5) 

by  Equations  (4.1a)  and  (4.3a).   The  error,  y (n) ,  is  by 
definition  the  accumulated  error  in  the  states  of  the  finite 
precision  realization.   In  order  to  find  an  equation  for  the 
propagation  of  this  error.  Equation  (4.5)  can  be  rewritten  as 

y(n)  =  Ax(n-l)  +  Bu(n-l)  -  [Ax«(n-l)]q  -  [Bu(n-l)]q 
+  Ax*(n-1)  -  Ax*(n-1) 
=  A[x(n-1)  -  x*(n-l)]  +  Ax*(n-1)  -  [Ax»(n-l)]q 
+  Bu(n-l)  -  [Bu(n-l)]q 
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y(n)  =  Ay(n-l)  +  e(n)  (4. 6a) 

Thus,  the  accumulated  error  in  the  states  at  time  nT  depends 
on  the  previously  accumulated  error  (at  time  (n-l)T)  and  the 
error  Introduced  by  quantization  at  time  nT. 

Applying  Equation  (^.6a)  recursively  and  assuming  that 
there  Is  no  accumulated  error  at  time  nT  =  0,  I.e.  x*(0)  =  x(0), 
then  the  dependence  of  y(n)  on  all  previously  Introduced 
quantization  errors  can  be  found  to  be 


n 
y(n)  =   Z  a""  ^e(j)  (il.6b) 

J  =  l 


or  by  a  change  of  variable  In  the  summation. 


n-1 
y(n)  =   Z  A^e(n-l)  (iJ.6c) 

1=0 


Equations  (4.6a-c)  show  that  the  effect  of  quantization 
errors  Introduced  "long  before"  time  nT  becomes  small  since 
the  filter  is  assumed  to  be  stable  and  the  characteristic 
equation  of  the  filter  depends  solely  on  the  matrix  A.   What 
may  be  considered  "long  before"  depends  strongly  on  the 
characteristic  of  the  filter.   In  particular,  for  second 
order  filters,  a  low  damping  coefficient  would  imply  a  con- 
siderable time  for  the  effect  of  a  quantization  error  to 
become  negligible.   This  idea  will  be  useful  in  evaluating 
the  results  of  the  derivation  and  will  be  seen  to  be  applicable 
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to  the  stochastic  formulation  In  Section  C  as  well  as  In 
the  deterministic  Equation  (^.6). 

Defining  Av(n)  as  the  difference  between  the  outputs  of 
the  finite  and  infinite  precision  realization,  a  similar 
derivation  yields 

Av(n)  =  v(n)  -  v*(n) 

=  Cx(n-l)  +  6u(n-l)  -  [Cx*(n-1)]  -  [6u(n-l)] 
+  Cx*(n-1)  -  Cx»(n-1) 

=  Cy(n-l)  +  e(n) 

n-1     . 
=  C  E  A''~^~^e(j)  +  e(n) 
j=l 

n-2  . 
=  C  Z  A^e(n-l-l)  +  e(n)  (A.6d) 

1=0 

The  quantity,  Av(n)  is  the  accumulated  error  at  the  out- 
put of  the  digital  filter  at  time  nT .   Equation  (^.6d)  shows 
that  this  accumulated  error  depends  on  all  previous  quanti- 
zation errors  generated  In  the  state  equations  (not  including 
the  errors  generated  at  time  nT)  but  Av(n)  only  depends  on 
the  present  quantization  errors  generated  in  the  output 
equation . 
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Equations  (4.6)  apply  to.  the  S  forms  of  the  state  equa- 
tions.  In  order  to  do  the  same  for  the  T  forms  it  Is  neces- 
sary to  redefine  the  quantization  errors  In  the  output 
equation  at  each  time  nT  as 

e'(n)  =  C'x*(n)  -  [c'x*(n)]   +  5'u(n-l)  -  [6'u(n-l)] 

(4.7) 

Keeping  the  definition  of  e(n)  the  same  as  (4.4a)  except 
to  replace  B  by  B'  the  accumulated  state  and  output  error 
equations  become 


n    .       n-1  . 
y(n)  =  Ay  (n-1)  +  e(n)  =   Z  a"  ■Je(j)  =   E  A^e(n-l) 

j=l  1=0 

(4.8a) 


and 


Av(n)  =  C'y(n)  +  e' (n) 


n 
=  C  S  A'^"Je(j)  +  e'(n) 

n-1  . 
=  C  Z  A^e(n-i)  +  e' (n)  (4.8b) 

1=0 


Equation  (4.8a)  is  identical  to  Equations  (4.6a-c)  but 
(4.8b)  has  a  subtle  difference.   This  difference  is  that,  for 
the  T  forms,  the  output  error  depends  on  all  previously 
generated  errors  in  the  state  equations  including  the  most 
recent  (i.e.  at  time  nT) .   The  error  Av(n)  still  only  depends 
on  the  most  recent  error  e'(n)  of  the  output  equation. 
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The  difference  in  the  expressions  for  Av(n)  for  the  S 
and  T  forms  is  a  result  of  the  choice  of  state  and  output 
equations  and  will  present  no  difficulty  in  the  stochastic 
development  which  follows. 

C.   STOCHASTIC  FORMULATION  WITH  CORRELATION 

A  general  expression  for  the  variance  of  the  output  error 
when  the  quantization  errors  are  assumed  to  be  discrete  ran- 
dom processes  can  be  developed  by  making  use  of  the  error 
propagation  equations  of  the  previous  section.   This  expres- 
sion can  then  be  used  for  analyzing  the  specific  cases  of 
the  second  order  filter  and  particularly  the  canonic  forms 
of  Chapter  III. 

1 .   General  Formulation 

Consider  first  the  T  forms  of  the  state  equations  and 
thus  Equations  (4.8)  as  expressions  for  the  accumulated 
state  and  output  errors.   Assuming  for  the  present  that  all 
random  variables  and  processes  are  zero  mean  the  variance  of 
the  output  error  can  be  expressed  as 


a/^(n)  =  E[(Av(n))(Av(n))*]  (^1.9) 


where  E[-]  is  the  expected  value. 

By  applying  the  first  equality  of  Equation  (4.8b), 
the  output  error  variance  becomes 
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o/^(n)  =  E[(C'y(n)  +  e • (n) ) ( (e • (n) )^  +  y^(n)(G')^)] 

=  C'Ay(n)(e')^  +  C  E[y(n)(e'(n))^] 

+  E[e«(n)y^(n)](C')^  +  E[e ' (n) (e ' (n) )^] 

=  C'Ay(n)(C')^  +  2C'  E[y(n)e'(n)]  +  o  ^,  (n) 

(4.10) 


where 


Ay(n)  =  E[y(n)y^(n)]  (4.10a) 


2  2 

o^(n)    =  E[(e'(n))  ]   for  the  single  output  case 


(4.10b) 


and 


E[e'(n)y*(n)](C')^  =  C 'EEy (n) e ' (n) ]  (4.10c) 

Equation  (4.10c)  arises  when  a  scalar  output  is 
assumed;  thus  the  cross  terms  in  (4.10)  are  combined  in  the 
final  expression  of  (4.10). 

A  (n)  is  the  covariance  matrix  of  the  accumulated 
state  errors  at  time  nT.   The  dependence  of  ^^(n)    on  the 
statistics  of  the  quantization  errors  is  found  by  substituting 
(4.8a)  into  (4 .10a) .   Then 
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n  „  .       n 


A  (n)  =  E[{  Z    A''"^e(i)}{  I    e^(  j  )  (A^)"""-^ }  ] 


y         1=1  J  =  l- 


n   n 


=  I        Z   A''~^E[e(i)e*(j)](A^)''-<^  (il.ll) 

1=1  j=l 

Defining  a  quantization  error  correlation  matrix  as 


R^(n,m)  =  E[e(n)e^(m)]  (4.12) 


enables  (4.11)  to  be  written  as 


A^(n)  =   I   I   a'^-^R  (i,j)(A^)''  ^  (4.13) 

y      1=1  j=l     ^ 


It  is  now  possible  to  write  the  expression 


AA  (n)At  =   S    i   A''-^-'\(l,j)iA'')''-^^^ 

y  1=1  j=i 

n-l  n-1  ^  .  1-  n  T 

=   E    Z  a'^'^R  (i+l,J  +  l)(A^)''"-^  (4.14) 

1=0  j=0     ^ 

Subtracting  (4.14)  from  (4.13)  yields 


A  (n)  -  AA  (n)A^  =  R^(n,n)  -  aV  (1,1)  (A^)""       (4.15) 
«y         «/ 

I   R  (n,j)(Ab''~^  -  I    aX(1'J+1)(A^) 
j=l  ^  J=l 


+  "^V-^R  (i,n)  -  "e  A"-^R^(i  +  l,l)(A^) 
1=1      ^        i=l 


+  "e^  ''zV-^[R,(i,J)  -  R.d+lJ+DKA^'' 
1=1   J-1      ^ 
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Equation  (^.15)  can  be  simplified  if  it  is  assumed 
that  all  the  processes  involving  quantization  errors  are  wide 
sense  stationary  and  mutually  wide  sense  stationary.   Under 
this  assumption,  the  correlation  matrix  R  (n,m)  is  a  function 
only  of  the  difference  of  its  arguments,  (m-n) (taken  for 
notation  purposes  in  that  order) .   Then  the  correlation 
matrix  can  be  written 


R  (n,m)  =  R  (m-n)  (^.l6) 

e         e 


Under  the  assumption  of  wide  sense  stationarity , 
the  first  and  second  terms  of  (4.15)  contain  Rg(0) .   Further- 
more, the  double  summation  term  becomes  the  0  matrix,  so  that 
(4.15)  reduces  to 


A  (n)  -  AA„(n)A^  =  R^(0)  -  A'^R{0)(k^)'' 
y        y         e         e 


+  I   R  (j-n)(Ab''-^  -   I   aX(J)(^-  ) 
j=l  ^  J=l 


+   Z  a"-^R  (n-i)  -  I   pj^   ^R^(-i)(A^) 
i=l     ^        i=l 


(4.17) 


Now  consider  the  entries  of  the  correlation  matrix. 
If  e.(n)  and  e.(m)  are  the  i   and  j    components  of  e(n) 
and  e(m)  respectively  then  the  ij^"^  entry  of  the  correlation 
matrix  is 
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[Re(n,m)]^j  =  [E[e(n)e*(m) ] ]^j  (A.I8) 


ej^(n)e.(m)  =  e.(in)e^(n) 


[E[e(m)e^(n)]]..  =  [R  (m,n)]., 


This  implies  that  the  transpose  of  the  correlation  matrix 
is  the  correlation  matrix  with  its  arguments  reversed.   For 
the  case  of  mutually  wide  sense  stationary  processes  this 
means 


Rg(k)  =  RgC-k)  (i|.19) 


Making  use  of  (^.19)  in  (4.17)  it  can  be  written 
that 


A  (n)  -  AA  (n)A^  =  R  (0)  -  a'^R^  ( 0)  ( A^  )'^  (^1.20) 

y       y       e       ^ 

+  ^l    {A""^R^(n-i)  +  CA^-^R^(n-i)]^} 
i  =  l 

-  ''z\A^-iR  C-i)(A^)"  +  [A^-^R  (-i)(AS^]h 
i=l       ^ 

Now  as  n  becomes  large  the  last  summation  of  (4.20) 
becomes  negligible  since  the  filter  is  assumed  to  be  stable 
which  implies  that  lim  a"  =  0_.   The  second  term  in  (4.20) 
also  becomes  negligible  so  that  in  the  limit 
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Ay  -  AAyA^  =  Rg(0)  +  I    (A^R^d)  +  [A^Rg(l)]^}     (4. 21a) 


or 


oo 

A„  -  AA^A^  =  S  (0)  +   Z  {A^R^(i)  +  S  (-i)(A^)^}    (i|.21b) 


y    y     e   .  i=i 


The  left  hand  side  of  Equations  (4.21)  can  be  solved 
for  A   so  that  these  equations  essentially  give  an  expression 
for  A  ,  the  covariance  matrix  of  state  errors,  in  terms  of 
the  correlation  matrices  of  the  quantization  error  sources 
for  large  n  under  the  assumptions  of  zero-mean,  wide  sense 
stationary  and  mutually  wide  sense  stationary  error  processes. 


XiiC   ^  ^  J- 


u  ^  i  w  n  1/  o 


A   -  AA  A^  =  F  (4.22) 


(where  F  is  a  symmetric  matrix  which  represents  the  right 
hand  side  of  (4.21))  can  then  be  substituted  into  (4.10)  as 
part  of  the  solution  for  the  output  error  variance. 

The  factor  E[y(n)c'(n)]  which  also  appears  in  (4.10) 
must  be  found  before  the  output  error  variance  can  be  written 
solely  in  terms  of  the  statistics  of  the  quantization  error 
sources.   From  (4.8a)  it  follows  that 


n-1  . 
E[y(n)e'(n)]  =   I  A^E[e(n-i)c ' (n) ]  (4.23) 

1  =  0 


15^ 


Now  define  a  cross  correlation  matrix  (In  this  case 
a  vector)  betv:een  e(n)  and  e'(m)  as: 


R   ,(n,m)  4  E[e(n)e'(m)]  (i}.24) 


Under  the  assumption  of  wide  sense  statlonarlty ,  as 
before,  the  cross  correlation  vector  depends  only  on  the 
difference  In  Its  arguments.  I.e. 


R   ,(n,m)  =  R   ,(m-n)  (^1.25) 

ee  ee 


so  that  (4.23)  becomes 


E[y(n)£'(n)]  =   E  A^R^  .(1)  (4.26) 

1=0   ^^ 


and  It  follows  that 

00      .  ^ 

11m  E[y(n)e'(n)]  =   ZA^R^^,(1)  (4.2?) 

n-^<»  1=0 

Substituting  (4.27)  and  the  solution  to  (4.22)  into 
(4.10)  yields  the  output  error  variance  for  the  T  forms, 

a,^(e,e')  =  C'A,(e)(C')^  +  2C'  Z  A^R    (1)  +  o /,  ^^ '^^^ 

Av  —        y  ~"         1=0 

as  a  function  of  the  statistics  of  e  and  e' . 

For  the  S  forms,  the  difference  in  the  output  error 
propagation  equation  (4.7c)  causes  a  slight  difference  in  the 
expression  for  the  output  error  variance. 
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Substituting  (4.7c)  into  (4.9)  yields 


a/  (n)  =  E[(Cy(n-l)  +  e(n))(e'^(n)  +  y^(n-l)c^)] 


=  CA  (n-l)C^  +  2C  E[y(n-l)e(n)]  +  a^       (4.29) 


The  first  term  of  (4.29)  contains  A  (n-1)  instead  of  A  (n) 
which  appeared  in  (4.10).   But  in  the  limit 

A  (n-1)  =  A  (n)  =  A  (e).   The  second  term  of  (4.29),  however, 
results  in  a  slightly  different  expression  in  the  summation 
form.   Using  (4.7b) 


n-2 
E[y(n-l)e(n)]  =   E  A^E[e(n-l-i)e(n) ] 

1=0 

n-2  .  /s 
=  I   A^R  ^(i+1)    •       '         (4.30) 
1=0   ^ 


which   in   the    limit   becomes 


11m  E[y(n-l)e(n)]   =      EA^R^^(i+l)  (4.31) 

n-voo  1=0 


Then  the  expression  for  the  output  error  variance  for  the 
S  forms  is 


o  1-  i  "  2 

a/(e,e)  =  CA  (e)C^  +  2C  E  A^-R^^d  +  l)  +  a         (4.32) 
Av^-'       y  ^^Q   ee         e 

Equations  (4.28)  and  (4.32)  together  with  (4.21)  are 
the  principal  results  of  this  section.   They  express  the  output 
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error  variance  of  the  S  and  T  forms  In  terms  of  the  statistics 
of  the  Internally  generated  quantlza-clon  errors  Including  the 
effects  of  correlation.   The  assumptions  made  were  that  all 
processes  had  zero  mean  and  were  wide  sense  stationary  and 
mutually  wide  sense  stationary. 

When  the  assumption  of  zero  mean  processes  Is  relaxed 
the  expressions  do  not  change  appreciably.   All  that  is 
necessary  is  to  redefine  the  covariance  and  correlation 
matrices: 


A  (n)  =  E[{y(n)  -  m(y)}{y(n)  -  m(y)}*] 


=  E[y(n)y^(n)]  -  m(y (n) )m^ (y (n) ) 


=  A  (n)  -  m(y(n))ra^(y(n))  (4.33) 

•7 


R  (n,m)  ^  E[{e(n)  -  m(e (n) ) } {e (m)  -  m(e(m))}  ] 
=  E[e(n)e^(m)]  -  m(e(n) )m^ (e(m) ) 
=  R  (n,m)  -  m(e(n))m^(e(m))  (4.3^) 


R   (n,m)  ^  E[{e(n)  -  m(e (n) ) } {e (m)  -  M(e(m))}] 
ee   '        —      


=  E[e(n)G(m)]  -  m(e (n) )y (c (m) ) 


=  R   (n,m)  -  m(e(n))u(e(m))         (4.35) 
ee        
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where  m  and  y  are  the  means  of  their  arguments. 

When  it  is  assumed  that  the  processes  are  v;ide  sense 
stationary  it  is  possible  to  write 


\(e)  =  Ay(e)  -  m(y)m*(y)  (4.36) 


Rg(n,m)  =  R^(m-n)  =  R^Cm-n)  -  m(e)m^(e)      (^.37) 


Rg^(n,m)  =  ^^.(m-n)  =  R^^{m-n)    -   m(e)y(e)     (4.38) 


where  the  dependence  on  a  time  index  has  been  removed  because 
wide  sense  statlonarity  implies  dependence-  only  on  time 
differences . 

Then  the  expressions  for  the  output  error  variance 
for  the  S  forms  is 

^/v^-'^^  "  CA  (e)C^  +  2C  Z  A^R^^(i+l)  +  0^  (4.39) 

and  for  the  T  forms 


a.^(e,e')  =  C'i  (e)(C')^  +  2C'  Z  A^R    (1)  +  o^         (4.40) 

AV  y  -irrO 


where  A  (e)  is  the  solution  to 
y  - 


A„(e)  -  AA,  (e)A^  =  R^(0)  +  I    {A^R  (1)  +  [A^R  (1)]^} 
y  -      y  -       e      j_^-[_    e         e 


(4.41) 
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and 


al   =  E[(£  -  y^)^]  =  ECe^]  -  y^  m,l\2) 


These  equations  (('1.39)  through  (4.^12))  give  the 
general  expression  for  the  output  error  variance  assuming 
only  wide  sense  statlonarlty .   The  effects  of  structure  are 
Imbedded  In  the  nature  of  R  (1),  K   (1)  and  a      and  in  the 
entries  of  the  matrices  of  the  state  and  output  equations. 
The  next  section  examines  these  effects. 

a.   Structural  Effects  (General) 

In  order  to  take  topological  or  structural  effects 
into  account,  it  is  necessary  to  consider  the  terms  of  the 
correlation  matrices  of  error  sources  as  given  by  (^.l6)  and 
(4.25).    When  zero  correlation  is  assumed,  R  (k)  is  taken 
to  be  zero  for  all  k  5^  0 ,  and  R  (0)  is  taken  to  be  a  diagonal 
matrix  where  the  k   entry  on  the  diagonal  is  the  sum  of  the 
second  moments  (variance)  of  each  individual  error  source 
comprising  e,  (n) .   Assuming  uniform  distributions,  these 
terms  become-^  (a  )  times  the  number  of  unique  non-zero  non- 
unity  multipliers  in  the  k   row  of  the  matrices  which 
contribute  to  the  error  ©^(n) . 

When  correlation  is  assumed  (both  spatial  and 
time)  the  picture  becomes  much  more  complicated.   In  general. 


This  development  returns  to  the  assumption  of  zero  mean 
processes  for  ease  of  notation  since  there  is  no  loss  of 
generality  in  doing  so. 

^a^  =  h^/12  for  rounding,  h'^/S  for  truncation. 
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,th 


the  (IJ)    term  of  the  correlation  matrix  is  given  by 


[Re(n, 


m) 


IJ 


E[e(n)e^(m)] 


j,j  =  E[e^(n)ej(m)] 


=  E 


Pi  qj 

^    ^  e.  (n)e.„(m) 
k=i  Jl=i  Ik    JJi 


Pi   qj 


(i».il3) 


where  p^  is  the  number  of  multiplier  errors  contributing  to 
^1'  "^i  ^^   ^^^  number  of  multiplier  errors  contributing  to 
e. ;  and  k,^  are  additional  Indices  to  identify  the  contribu- 
tions to  e. ,  e .  respectively;  that  is 


P.- 


^J 


e^in)    =   Z  e^^(n)   and  e.(n)  =   Z  e  j^(n)     (h.H'4) 


k=l 


£=1  J 


In  all,  then,  there  are  six  indices  to  be  considered  to 

determine  the  entries  of  the  covarlance  matrix;  two  (i,j) 

associated  v/ith  states;  two  (k,£)  associated  with  multipliers; 

and  two  (njm)  associated  v/ith  time.   When  wide  sense  statlon- 

arity  is  assumed,  then  one  index  representing  time  difference 

replaces  the  two  indices  of  time. 

Considering  a  typical  contribution  to  R  (n,m), 

such  as  E[e.,  (n)e.(,(m)],  the  question  to  be  asked  is  when  is 
Ik     J  Jo 

this  correlation  statistic  non-zero?  One  obvious  answer  is, 
when  the  multiplier  coefficients  are  the  same  number  and  the 
signal  quantities  are  identical  (e.g.,  XpCn)  =  x-,(n-l)).   In 
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this  case  there  Is  a  unity  correlation  coefficient  and 


E[e^j^(n)ej^(m)]  =   ECe^^^Cn)  ]E[ej2^(m)  ]    .    (4.il5) 


Another  possibility  to  be  investigated  is  when 
the  signal  quantities  are  identical,  but  the  multiplier 
coefficients  are  different.   If  the  signal  quantities  have 
Gaussian  distributions  then  the  correlation  coefficient  is 
a  function  of  the  ratio  of  the  two  coefficients  as  shown  in 
Appendix  B  .   As  expected  in  this  case  if  the  multiplier 
coefficients  are  the  same,  the  error  statistics  have  unity 
correlation.   Note  that  the  correlation  coefficient  depends 
on  the  sign  of  the  ratio  of  the  multipliers. 

V/hen  the  m.ultiplier  coefficients  are  the  same 
but  the  signal  quantities  are  not  identical,  then  it  is 
suggested  that  the  correlation  of  errors  will  depend  mainly 
on  the  correlation  between  the  signal  quantities. 

Finally,  the  most  complicated  case  is  when  the 
coefficients  are  different  and  the  signal  quantities  are 
not  identical.   It  is  expected  in  this  case  that  the 
correlation  is  small. 

The  foregoing  discussion  applies  equally  well 
to  the  terms  com.prising   R   (n,m).   Experimental  determina- 
tion  of  correlation  effects  are  considered  in  Appendix  D 
for  second  order  canonic  filters.   In  the  next  section  the 
general  error  variance  equations  are  simplified  for  the 
second  order  filter. 
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2.   Simplification  tc  the  General  Second  Order  Section 
a.   For  Arbitrary  State  Equations 

For  the  second  order  filter  the  solution  to  (4.22) 
is  a  symmetric  matrix 


A  = 

y 


^11    ^12 


^21    ^22 


(i|.i|6) 


where 


\^^   =  {ril[(l-a,,a22)(l-a2/)  -  ^^2^21^^'-^22^^ 


+   2f^2^aja2ia22   +   a^^a^g^l-a^^)  ] 


(4.il6a) 


+    f22^^1^^11^22   -   ^12^21^^12^^/^ 


2  2 

^Yno    "    \..    "    ^^ll^^'^"^22^^11^21    ^    ^12^2l''22^ 
"^12  "^21 


+   f^^L{l-a^l){l-a/^)    -   aja2i] 


+    f22^^^-^ll)^12^22   ^    ^lA2^21^^/^ 


(i<.46b) 


^^22   ^    ^^11^^^''^11^22   -   ^12^21^^21^ 


+    2f^2^a^^a^2^21   "^   a^^a^gd-a^^)  ] 


(4.46c) 


+   f22[(l-aiia22)(l-ai^)    "   a^2^2l(^-^^ll^  ^  ^^^ 
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with 


A  = 


(l-aj)(l-aj)(l-a^^a22  "  ^12^21^ 


2(l-a^2)a^2^2A2 


2(l-a22)a^^a^2^21 


^■^~^11^22"^12^21^  ^12^21 


^^11^12^21^22 


i^.^Sd) 


and 


P  = 


f     f 
11     12 


f      f 
21     22 


With  f^2  =  ^21 


(i|.46e) 


For  general  second  order  state  equations  this 
solution  could  then  be  substituted  into  (4.28)  or  (4.32). 
b.   For  Canonic  Arrays 

When  the  state  equations  are  specifically  those 
of  the  canonic  arrays  in  Chapter  III,  it  is  possible  to 
substitute  the  coefficients  found  there  into  Equations  (4.46) 
to  find  A   in  terms  of  those  coefficients  and  the  entries  of 

y 

the  matrix  F.   If  the  factor  E[y (n-1) e (n) ]  in  (4.29)  is 
given  by 


-  E[y(n-l)e(n)]  =   Z  A^R^^(iH-l)    (4.4?) 


1  =  0 


ee 
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then  the  expression  for  the  output  error  variance  for  the  S 
forms  of  the  state  equations  become: 


SM:  a^^   =  {f^^[(c^  +  e^)(l  +  b)  +  ce(-2a)] 


+  2  ^-^2^{o'^   +  e^)(ab)  +  ced-a^-b^)] 


+  f22[c^b^(l+b)  -  2ce(ab^)  +  e^[(l+b)  -  a^(l-b)]}/A 


+  2(cg^  +  eg^)  +  a^  (i].il8a) 


SM^:  o^^   =  {f^^(l+b)  -  2af^2  +  ^^2^'^^^^^^^  (4.il8b) 


'Av    '11 


+  2f^2t(^^  "^  e^)(l+b+ab)  +  ce  ( l-a^-b^-2a)  ] 


+  f22[c^{2(l+b+ab)-(l+b)^(l-b)} 


+  2ce{(l-b)(l+b+ab)-a(l+a+b)} 


+  e^{2(l  +  b  +  ab)  -  a^(l-b)}]}/A 


+ 


2{c(g^  +  g2)  +  eCcg))  +  ol  {^Mc) 


,t    2 


SN^:  a^^   =  {2f3_^(l+a+b)  -  2i:^^{l^a.^h)    +  r^^il+h)]/^ 
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where 


A   =  (l-b)[(l+b)2  -  a^] 


(4.ii8e) 


When  the  T  forms  of  the  state  equations  are  used, 
let  the  factor  E[y(n)e'(n)]  in  (^1.10)  be  given  by 


G'  = 


61  • 


=  E[y(n)e'(n)]  =   I  A^R   .  (i)    (i|.49) 

1=0   ^^ 


For  the  T  forms,  the  expressions  for  the  output  error  variance 
are  identical  to  Equations  (4.48)  for  the  coresponding  canonic 

arrays  (e.g.,  TM   corresponds  to   SM  ,  TM   to  SM   ,  etc.) 

2 
except  that  c,  e,  g  ,  g^  and  a   are  replaced  by  c',  e',  g-.  ' , 

2 
gp'  and  a    ,,  respectively. 

Note  that  the  transpose  forms  do  not  apparently 

depend  on  c  and  e.   The  factors  f. .  however  do  depend  on 

these  coefficients  since  in  the  transpose  case  they  appear 

in  the  state  equations  rather  than  the  output  equation. 

The  quantization  errors  they  cause  will  appear  in  e_(n)  thus 

in  R  (1)  and  thus  in  f . . . 
e   '  ij 

The  output  error  variance  for  the  transpose  cases 
does  not  however  depend  on  any  G  or  a   since  e(n)  (ore'(n)) 
is  zero  for  the  transpose  case,  there  being  no  non-zero, 
non-unity  multipliers  in  the  output  equation. 

The  expressions  for  the  output  error  variance 
given  in  Equations  (4.48)  are  identified  without  subscripts 
but  rather  by  an  identification  v.-hich  Implies  the  applicability 
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of  that  expression  to  all  members  of  the  set  of  realizations 
with  that  basic  identification  (e.g.  SN^ „  is  a  member  of 
the  set  of  SN  realizations).   The  reason  these  expressions 
apply  to  the  entire  set  is  that  the  development  to  this  point 
has  depended  solely  on  the  basic  state  equations.   The  dif- 
ferences of  structure  implied  by  the  difference  between  the 
eight  basic  forms  of  the  state  and  output  equations  exhibit 
themselves  in  the  four  basic  expressions  of  (4.48)  for  the 
S  forms  and  the  same  four  expressions  for  the  T  forms.   The 
differences  of  structure  imposed  by  the  various  manipulations 
of  the  flow  diagrams  are  entirely  embodied  in  the  factors 
f . .  and  g. .   What  remains  to  be  examined  is  how  that  structure 
exhibits  itself  in  those  factors.   Structural  effects  were 
discussed  in  general  in  Section  C2.   Now  they  will  be  examined 
In  the  context  of  the  canonic  arrays  starting  with  the  simpli- 
fying assum.ption  of  uncorrelated  error  sources.   This  condition 
will  then  be  relaxed  to  achieve  some  better  Insight  into  the 
effects  of  correlation. 

c.   Uncorrelated  Case 

When  all  the  error  sources  are  assumed  to  be 
uncorrelated,  the  right  hand  side  of  Equation  (4.22)  becomes 


P 


E[e2(n)] 


E[e^(n)] 


(4.50) 


.66 


2 
where  E[e^(n)]  Is  the  sum  of  the  variances  of  the  individual 

multiplier  errors  contributing  to  e.(n). 

For  example,  in  the  realization  of  the  array, 

SM^Q,  the  first  state  equation  is 


x^(n)  =  c(x^(n-l)  +  u(n-l))  -  (a+c)x^(n-l)  +  X2(n-1) 

(4.51) 
therefore 

e^(n)  =  c{x*(n-l)  +  u(n-l)}  -  [c{x*(n-l)  +  uCn-l)}]^ 

+  {-(a+c)x*(n-l)}  -  [{-(a+c)x*(n-l)}]q 


+  x*(n-l)  -  [x|(n-l)]q 


=  ^c^^)  ■"   ^a+c)^-)  -^  °  ^^^-52) 


and 


ECe^(n)]  =  E[{e^(n)  +  e^^^^^Cn)}^] 

since  E[e  (n)e,  ^  An)']   is  assumed  to  be  zero  because  the 
c    (a+c; 

error  sources  are  assumed  to  be  uncorrelated. 
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If  one  assumes  that  all  error  sources  are 
identically  distributed  with  variance  a^,  then  for  the 
example  SM   the  variance  of  e,  (n)  is 


E[e^(n)]  =  2a2 


(4.54) 


In  general,  for  the  uncorrelated  case 

;2 


F  = 


y^cr 


0 


0 


M^o 


(4.55) 


where  y^  (i  =  1,2)  is  the  number  of  unique  non-zero  or 
non-unity  multipliers  in  the  i   row  of  the  canonic  array. 
For  the  example,  the  array  is 

c-(a+c)    1     c 

-b      0     e  (4.56) 

1      0     d 


SM 


20 


It  appears  at  first  that  there  are  three  non-zero, 
non-unity  multipliers  in  the  first  rov;.   But  by  referring 
to  the  flow  diagram  of  this  realization  (Figure  3-20)  or 
Equation  (4.51),  it  is  seen  that  there  is  only  one  multiplier 
with  c  as  the  coefficient  and  one  with  a+c.   Thus  there  are 
tv/o  quantizations  required,  one  after  each  of  these  two 
multiplications . 
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SM 


In  the  array 

c-(a+c)  -b  1 

1  0  0 

c  e  d 


20 


(^.57) 


there  are  three  unique  non-zero,  non-unity  coefficients  in 
the  first  row  and  none  in  the  second.   Then  for  SM, 

F  =  (4.58) 


'20 


0 


0 


0 


whereas  for  SM 


F  = 


20 


2a 


2a' 


Therefore  in  counting  coefficients  for  forming  F,  only 
unique  non-zero,  non-unity  coefficients  in  each  row  are 
included,  because  the  same  reasoning  is  true  in  all  the 
realizations. 

To  find  the  entries  of  the  vector  G  or  G' ,  it 
is  necessary  to  examine  the  relationships  between  the  state 
equations  and  the  output  equation. 

From  Equation  (4.^7)  and  the  definition  of  R   (k), 


G  =   E  A  E[e(n-i-l)e(n)] 
1=0 


(^.60) 
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and  from  (4.49) 


G'  =  I   A^E[e(n-l)e'(n)]         '  (4.6l) 

1=0 


At  first  glance.  Equations  (4.60)  and  (4.6l) 
would  appear  to  be  zero  when  error  sources  are  assumed  to 
be  uncorrelated.   But  It  Is  to  be  remembered  that  e(n)  and 
e(n)  are  not  error  sources.   Rather  they  are  composite  errors 
made  up  of  the  errors  from  possibly  several  sources.   In 
fact.  In  som.e  of  the  realizations  of  Chapter  III  a  particular 
source  of  error  may  find  Its  way  into  a  component  of  e(n) 
and  also  into  e(n). 

Consider  for  example  the  flow  diagram  for  the 
realization  TM  (see  Figures  3-20  ).   The  first  state  equation 
for  this  realization  is 

x^(n)  =  -ax^(n-l)  -  bx2(n-l)  +  u(n-l)        (4.62a) 

while  the  output  equation  is 

v(n)  =  c'x^(n)  -  ax2(n)  +  (a+e')x2(n)  +  d'u(n-l) 

=  c'Xj_(n)  -  ax^(n-l)  +  (a+e')x2(n)  +  d'u(n-l) 

(4.62b) 

The  error  e-,  (n)  in  this  case  represents  the  sum 
of  the  two  errors  from  quantizing  ax^(n-l)  and  bx2(n-l)  while 
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e'(n)  Is  made  up  of  the  errors  from  quantizing  c'x^(n), 
(a+e')xp(n)  and  ax  (n-1) ,  so  that 

e^Cn)  =  e^(n)  +  e^(n)  (4.63a) 


and 


e'(n)  =  e^,(n)  +  e^^^^.^Cn)  +  e^(n)  (4.63b) 


Then 


E[e^(n)e'(n)]  =  E[e^(n)e^ , (n) ]  +  E[e^(n)e(^^^ , ^ (n) ] 


+  E[ef(n)]  +  E[e. (n)e^,(n)] 


+  E[e^(n)e(^^g,)(n)]  +   E[e^(n)e^(n) ] 

(4.64a) 


Under  the  assumption  of  uncorrelated  error  sources  all  these 

2 

terms  are  zero  except  E[e  (n)]  so  that 


E[e,(n)e'(n)]  =  E[ef(n)]  7^  0  (4.64b) 

X  a 


This  implies  that  in  Equation  (4.6l)  the  first  term  of  the 
summation  is  not  zero.  The  other  terms  are  zero,  however, 
in  the  uncorrelated  case. 

Upon  examination  of  all  realizations  it  turns 
out  that  only  TK..  and  TMp  fit  this  pattern.   All  transpose 
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configurations  require  that  e(n)(or  e'(n))  be  zero  since 
there  are  no  non-zero,  non-unity  multiplications  In  the  output 
equation.   For  the  SM  and  SN  forms,  although  In  most  cases 
there  are  coefficients  which  contribute  to  both  e(n)  and 
e(n),  the  term  E[e(n)e(n)]  does  not  appear  In  Equation  (4.60). 
TMq  and  TNq  have  no  coefficients  which  contribute  to  both 
e(n)  and  e' (n) . 

Again  assuming  Identically  distributed  error 

"2 
sources  with  variance  a    ,  then  for  the  realizations  TM-,  and 


G'  = 


81 

__ 

';2- 

.4. 

_   0  _ 

(i».65) 


Although  there  are  only  two  cases  when  G'  Is 
non-zero  for  uncorrelated  error  sources,  the  exercise  in 
finding  them  will  be  helpful  in  understanding  the  effect  of 
structure  when  correlation  is  considered. 

Finally,  the  last  term  of  (^^.28)  or  (4.32) 

Involves  the  variance  of  the  quantization  errors  contributed 

2       2 

by  the  output  equation  alone,  i.e.,  a   (or  a  ,).   When 

uncorrelated  error  sources  are  assumed,  this  term  is  the 
sum  of  the  variances  of  the  individual  errors.  Assuming 
identically  distributed  error  sources. 


2    "2 

o     =   vo 
e 


i^.e^a) 
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where  v  is  the  number  of  non-zero,  non-unity  elements  in 
the  output  row  of  the  array . 

Using  the  previous  results.  Equations  (^1.48)  for 
the  S  forms  become: 

SM,SN:    o^^   =  {(y^/A)[(c^+e^)(l+b)  -  2ace]  +  v}a2 

(4.66a) 

^^^'-  °^Av  "  ^^1  "^  Vi2)(l+b)a2/A  (4.66b) 


SN 


*••     cr^^^  =  {2y^(l+a+b)  +  v^{l+h))o'^ /h  (4.66c) 


where  A  is  defined  in  (4.48e). 

For  the  T  forms  these  equations  become 


TMq,TNq:    a^^^  =  {(y^/A)[((c')^  +  (e  '  )^)  (1+b  )~2ac  •  e  '  ]+v}a 


2 

(4.66d) 


TM^,TM2:    o  ^^  =    { (y^/A) [ ( ( c ' ) ^+(e ' ) ^) (1+b )-2ac 'e ' ]  +  2c ' +v la^ 

(4.66e) 
TM*:    0^^^  =  (y^  +  y2)(l+b)a^/A  (4.66f) 


TN^:    a^^   =  {2y^(l+a+b)  +  y2(l+b)}a^/A         (4.66g) 


The  fact  that  y   =  0  for  non-transpose  configura- 
tions is  already  included  in  Equations  (4.66). 

Table  4-1  lists  the  results  of  the  analysis 
above  for  each  canonic  array  for  the  uncorrelated,  Identically 
distributed  case. 
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ARRAY 

^1 

^2 

V 

S^OO'^^OO 

2 

0 

2 

s^^oi'S^io, 

SN 

01- 

,SN 

10 

2 

0 

3 

SMo2,SM2o, 

SN 

02^ 

,SN 

20 

3 

0 

2 

SM^2'2M2i, 

SN 

12^ 

,SN 

21 

3 

0 

3 

S^ll'SNii 

2 

0 

4 

SM22>SN22 

f 

1] 

0 

2 

<r™i' 

2 

2 

0 

SN^^.,TN.^. 

2 

2 

0 

TMq,TN^ 

2 

0 

2 

TM^ 

2 

0 

3 

TM^ 

3 

0 

2 

2 

a,  formula 


Av 


# 


(2p+2)a^  1 

(2p+3)o^  2 

(3p+2)a2  2 

(3P+3)a^  ^ 

(2p+4)a2  5 

(4p+2)a^  6 

4(l+b)a^/A  7 


[6(l+b)  +  ila]a^/A  8 

(2p'+2)a^  9 

(2p'+2c'+3)a^  10 

(3p'+2c'+2)a^  11 


p  =  [(c^  +  e^)(l  +  b)  -  2ace]/A 


pt  =  [((c')   +  (e')  )(1  +  b)  -  2ac'e']/A 


A  =  (1  -  b)[(l  +  b)^  -  a^] 


TABLE  n-1   FORMULAE  FOR  a^  FOR  UNCORRELATED,  IDENTICALLY 

^   .  -2 

DISTRIBUTED,  ERROR  SOURCES  (MEAN^O ;VARIANCE=a  ) 


17** 


Appendix  C  verifies  that  none  of  the  formulae 
for  the  uncorrelated  case  violate  the  requirement  that  they 
predict  a  non-negative  number  for  a^   .   Briefly,  the  results 
of  that  appendix  are  that  for  a  stable  filter 

^10  (^.67a) 

p  (or  p')  >  0         •                    (i».67b) 

P'  +  C  >  -  ^                             (4.67c) 

(l+b)/A  >  1  (4.67d) 

Examination  of  the  relative  magnitudes  of  the 

2 
expressions  for  a    shows  that  the  smallest  value  for  formu- 

lae  1  through  6  Is  (2p  +  2)a   and  from  (4.67b)  the  smallest 

this  can  be  Is  2a  .   Formula  7  Is  at  least  4a   by  (4.67d) 

^2 

and  formula  8  is  at  least  2a   since  a  Is  at  least  -(1+b). 

Formulae  9  through  11  predict  a  minimum  variance  of  2a  , 

2.50   and  1.51^   respectively. 

It  has  been  stated  empirically  [    ]  that,  of 

the  canonic  realizations  derived  by  Parker  and  Hess,  noise 

generation  Is  less  In  the  S,  (SN)  forms  than  In  S^(SM),  but 

u  a 

t    t 
that  the  transpose  forms  had  still  less,  S  (SN  )  being 

better  than  S  (SM  ),  I.e.  the  least  error  occurs  In  SN  . 

Table  4-1  does  not  predict  this,  nor  does  it  contradict  it. 

But  formula  1  predicts  that  SM^q  and  SN_q  yield  the  lov/est 
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noise  for  the  untransposed  S  forms,  making  no  distinction 
between  SM  and  SN.   Formula  8  says  any  of  the  SN^  configurations 
are  probably  as  good  as  SM^^  and  SN     Furthermore,  that 
same  formula  indicates  that  the  TN  forms  are  comparable 
and  formula  9  adds  TM^  and  TNq  to  the  group  with  a  minimum 
noise  of  2a  . 

The  most  Intriguing  suggestion  of  Table  ^-1  is 
that  TM2  may  yield  the  lov;est  noise,  having  a  minimum  variance 
of  l.Sa'^. 

But  all  this  is  highly  conjectural  especially 
since  only  minimum  values  are  being  compared.   There  are 
few  sets  of  coefficients  which  will  result  in  these  minima 
being  achieved.   For  example,  (l+b)/A  is  only  minimum  at 
the  origin  in  the  (a,b)-plane.   The  factors  p  and  p'  can  be 
minimum  for  any  value  of  a  .and  b  as  long  as  c  and  e  (or  c' 
and  e')  are  both  zero  which  is  a  degenerate  case. 

Furthermore,  this  analysis  vjas  based  on  the 
assumption  of  identically  distributed,  uncorrelated  error 
sources.   More  useful  and  interesting  results  can  be  achieved 
by  relaxing  these  conditions.   Furthermore  it  is  closer  to 
reality  as  will  be  seen  in  the  next  section. 
d.   Correlated  Case 

When  the  quantization  error  sources  are  assumed 
to  be  correlated,  the  quantities  f . .  in  Equations  (^.48) 
are  given  by  the  matrix  equation 
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F  -  R^(0)  +   Z  {A^S  (1)  +  [A^R  (i)]^} 


(il.68) 


i=l 


The  entries  of  Rg(i)  were  discussed  In  general 
in  Section  C-l-a  of  this  chapter.   It  was  shown  there  that 
these  entries  consisted  of  auto-  and  crosscorrelatlon  terms 
of  the  error  sources. 

For  example,  consider  the  realization  TM--  for 
which  the  array  Is 

-a    1    c' 

-b    0    e' 

1     0    d' 

and  the  structure  is  that  of  Figure  3-^2:.   The  error  vectors 
for  this  realization  are  given  by. 


e^(n) 
egCn) 


e^(n)  +  e^,(n) 
e^(n)  +  e^, (n) 


(4.69a) 


e(n)  =  0 


(4.69b) 


where  e  (n)  is  the  error  introduced  by  the  multiplication 
a 

of  X, (n-1)  by  a  and  the  subsequent  quantization  of  that  pro- 
duct at  time  nT.   The  errors  e^(n),  e^,(n)  and  eg,(n)  are 
similarly  defined.   Figure  4-la  shows  a  schematic  represen- 
tation of  the  error  sources  for  TM^q.   Figure  4-lb  shows  the 


177 


composite  errors  e, (n)  and  ep(n)  as  composite  error  sources 
where  e,(n)  and  e2(n)  are  given  by  (4.69a). 

The  correlation  matrix  Rg(l)  Is  given  by 


Rg(i)  =  E[e(n-l)e"(n)]  = 


E[e^(n-l)e^(n)]   E[e^(n-l)e2(n) ] 


E[e2(n-l)e^(n)]   E[e2(n-l)e2(n) ] 


Substituting  (4.69)  into  (4.70)  yields 


(4.70) 


Red)  = 


R^^(i)    R^2^^) 


R2i(i)    R22(i) 


(4.71) 


with 


R^j^(i)  ^  E[ej.(n-i)e^(n)] 


(4.71a) 


^ll^i)  =  \a(i)  ^  \c'(i)  ^  ^'a^i)  ^  ^c'c'(i)     ^^-71^^ 


R,,(l)  =  R^^(i)  +  R_,(i)  +  R.,K(i)  +  R..p.(i)    (^.71c) 


12 


'ab 


ae 


c'b 


c '  e 


^21^^)  =  ^ba^i^  ^  \c'^^^  ■*■  ^-'-^^^  "■  ^p'n'd)     (^-71^) 


e  'a 


R22(i)  =  Rbb^)  ^  ^be'^^^  ^  ^e'b^^)  ^  ^.'e'^^^     ^^'"^^^^ 


where 
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R^gCi)  =  E[e^(n-l)eg(n)]  (^.71f) 


under  the  assumption  of  wide  sense  statlonarlty . 

Appendix  D  describes  an  experiment  which  was 
conducted  to  check  the  validity  of  Equations  (^1.48)  for  the 
realization  TMqq.   The  "actual"  output  error  variance  for  a 
white  noise  Input  was  calculated  by  summing  the  square  of 
the  difference  between  the  outputs  of  a  double-precision 
realization  and  one  which  rounded  products  to  one  decimal 
point.   (This,  of  course,  required  the  assumption  that  the 
process  v;as  ergodic  (or  stationary  In  a  strict)  sense  in  order 
for  the  time  average  to  be  equal  to  the  ensemble  average.) 
During  the  process  of  rounding  products,  the  error  from  each 
quantization  was  saved,  correlations  were  calculated  and 
substituted  into  R  (1)  and  F  was  computed.   The  "predicted" 
output  error  variance  was  found  using  (4.^ 8b)  and  the  a 
posteriori  information  about  the  correlation  of  the  error 
sources.   Excellent  correspondence  between  the  actual  and 
predicted  values  was  observed  for  three  sets  of  transfer 
function  coefficients. 

Of  course  F  v;as  not  computed  as  an  infinite  sum, 
but  a  sufficient  number  of  terms  were  used  to  observe  the 
convergence.   Of  particular  interest  is  the  observation  that 
a  higher  damping  coefficient  caused  more  rapid  convergence. 
This  is  to  be  expected  since  it  is  the  nature  of  a  stable 
filter  that  as  the  power  of  the  matrix  A  becomes  larger  the 


179 


entries  converge  to  zero  and  that  the  rate  of  this  conver- 
gence depends  on  the  damping  coefficient.   But  each  term  of 
F  also  Includes  a  correlation  matrix.   The  experiment  shows 
that  convergence  to  within  0.1%  Is  faster  when  the  filter 
Is  driven  by  an  uncorrelated  Input  than  when  driven  by  a 
highly  correlated  one  such  as  a  step  (Heavlslde)  function. 
This  too  Is  to  be  expected  since  a  highly  correlated  Input 
Implies  higher  correlation  In  the  states  and  thus  higher 
correlation  In  the  error  sources.   This  tends  to  make  the 
entries  of  the  correlation  matrices  larger,  hence  a  slower 
convergence  of  F  Is  expected.   An  heuristic  examination 
of  this  Idea  follows. 

Consider  the  nature  of  the  quantization  error 
source.   rj-gure  h  —  tz.a.   b>n^^  <->    one    j.uih-i^j->-ii  l-^  \^^  j  j      ^  ^j-    ±  ^  i^iv^-Liig, 
where  f(n)  is  the  number  to  be  rounded  and  h  is  the  quanti- 
zation step  size.   The  dotted  line  represents  the  identity 
function  f(n)  =  f (n) .   Figure  4.2b  shows  the  error 
e[f(n)]  =  f(n)  -  [f(n)]  .   Note  that  the  error  can  be  written 
as 

e(n)  =  e(f(n))  =  f(n)  -  hk(n)  (4.72) 


with 


k(n)  =  [(f(n)  +  |)/h]3.  (^-73) 


where  [.]   means  the  largest  Integer  less  than  (•).   Equation 
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(11.73)    Implies   that 


(f(n)    -  |)/h     <   k(n)    <    (f(n)   +  h/h  (H.7H) 


which  Implies  that 


hk(n)  -  ^  <  f(n)  <  hk(n)  -f  ^  (^.75) 


2   ^''''    -   "'^^"^  '  2 


Now  consider  the  autocorrelation  of  the  error 


E[e(n-i)e(n)]  =  E[(f (n-i)-hk(n-i) ) (f (n)-hk(n) ) ] 


=  E[f(n-l)f(n)]  -  hE[k(n-i)f(n)  +  f(n-i)k(n)] 


+  h^E[k(n-l)k(n)] 


=  E[f(n-i)f(n)]  -  hE[f (n-l)k(n) ] 

-  h{E[k(n-l)f(n)]  -  hE[k(n-l)k(n)  ]}  (iJ.76) 

If  the  discrete  probability  density  function 
(p.d.f.)  of  the  random  variable  f(n)  is  given  by  Pp(f(n))  = 
Pr(F  =  f)  and  the  p.d.f.  of  k(n)  is  Pj^(k(n))  =  Pr(K  =  k) 
then  by  (k .7k)    and  (^.75) 


Pj^(k(n))  =  Pr[(f(n)  -  |)/h  <  k(n)  <  (f(n)  +  |)/h] 


=  P^,[hk(n)  -  I  <  f(n)  <  hk(n)  +  |]       (^^.77) 
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If  the  cumulative  distribution  function  (c.d.f.) 


of  f(n)  is  given  by  Pp(f(n))  then 


Pj^(k(n))  =  Pp(a2)  -  Pp(a^)  (^.78) 


where 


a^  =  hk(n)  -  I  (4.78a) 


a2  =  hk(n)  +  |  (4.78b) 


This  says  that  k(n)  is  distributed  in  such  a  way  that  its 
statistics  are  very  similar  to  those  of  f(n)  so  that  Equation 
(^.?6)  ].ooks  like  it  has  four  terms  that  are  very  similar 
in  form  to  E[f (n-i) f (n) ] ,  the  autocorrelation  of  f (n) .   That 
is,  it  appears  that  the  form  of  the  autocorrelation  of  the 
error  should  be  similar  to  that  of  the  autocorrelation  of 
the  number  to  be  quantized. 

In  Appendix  F  it  is  shown  that  the  autocorrela- 
tion of  the  state  variables  of  a  linear  filter  (i.e.,  an 
infinite  precision  linear  filter)  can  be  found,  given  the 
autocorrelation  of  the  input  and  the  transfer  function  between 
the  input  and  the  state  under  consideration.   In  the  previous 
paragraph  it  is  suggested  that  the  autocorrelation  of  the 
errors  is  related  very  close].y  to  the  autocorrelation  of  the 
signal  quantity  to  be  quantized.   In  all  cases  this  signal 
quantity  is  the  product  of  a  coefficient  and  either  a  state 
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or  the  input.   The  process  of  multiplying  by  a  coefficient 
does  not  effect  the  autocorrelation  of  the  signal  quantity 
before  or  after  the  multiplier  except  to  scale  it  by  the 
square  of  the  coefficient.   Therefore,  it  is  suggested  that 
the  autocorrelation  of  each  error  is  similar  in  form  to  the 
autocorrelation  of  either  a  state  variable  or  the  input. 
A  sim.ilar  argument  suggests  that  the  cross  correlation  of 
error  sources  is  related  to  the  cross  correlation  of  two 
signal  quantities  (or  to  the  autocorrelation  of  a  single 
signal  quantity  if  the  two  coefficients  multiply  the  same 
signal  quantity). 

For  simplicity,  two  extreme  cases  that  bound  the 
forms  of  correlation  funct~ions  will  be  considered:  (1)  when 
the  input  is  uncorrelated  and  (2)  when  the  input  is  highly 
correlated  (i.e.,  its  autocorrelation  is  a  constant). 

(1)   Reduction  for  Uncorrelated  Input.   Appendix 
F   shows  that  when  the  input  is  uncorrelated,  the  auto- 
correlation of  the  states  will  be  a  function  bounded  in 
magnitude  by  Ke"^'"'"^  where  K  is  the  variance  of  the  state 
and  Y  is  proportional  to  the  damping  coefficient  of  the 
transfer  function  between  the  input  and  the  state  under 
consideration.   But  by  Equations  (3-9)  and  (3.11)  it  can  be 
seen  that  the  characteristic  function  for  the  state  equations 
is  the  same  as  that  of  the  overall  transfer  function  so  that 
Y  is  the  same  for  both  states  and  output. 

The  cross  correlation  between  the  uncorre- 
lated input  and  each  state  is  proportional  to  the  unit  pulse 
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response  of  the  respective  state  and  the  cross  correlation 

between  the  two  states  Is  proportional  to  the  correlation  of 

the  two  unit  pulse  responses.   All  of  these  functions  are 

then  bounded  in  magnitude  by  an  envelope  proportional  to 
g-Y|n|T^ 

Assuming  the  correlations  of  the  error 
sources  behave  in  a  manner  proportional  to  the  signals  before 
the  multipliers,  these  correlations  may  be  written  as 

I«a6(^>l  lK„6e-^|l|T  (i,.79) 

where  R  qCD  is  defined  as  in  Cl.Ylf)  and  K  „  is  some  positive 

Otp  Cxp 

proportionality  constant  involving  the  variance  or  covariance 

of  the  errors  and  other  factors . 

It  may  be  possible  to  substitute  (^1.79)  into 

a  form  of  Equations  (4.4l)  which  involves  the  magnitudes  of 

the  factors  f, .  and  g. .   Then  to  find  If. . I  and  Ig.  i  one 
ij      ^1  ij        1 

could  examine 

<P>  <  <R  (0)>  +   E  {<A^><R  (i)>  +  [<A^><R  (i)>]^}  (4.80a) 
-    ^        i=l        ^  e 

<G>  <   Z  <A^><R   (i+l)>  (^.80b) 

i=0      ^^ 

00 

<G'>  <   Z  <A^><R   ,  (i)>  (ij.80c) 

-  i=0 

where  <   >  indicates  taking  the  absolute  value  of  each  entry 
of  the  matrix  contained  therein.   This  process  may  however 
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give  away  too  much  to  obtain  a  good  estimate  of  the  output 
error  variance.   This  avenue  is  not  pursued  here  but  is 
suggested  for  possible  future  study. 

(2)   Reduction  for  Highly  Correlated  Input. 
Appendix  F  shows  that  when  the  input  is  highly  correlated, 
the  autocorrelation  of  the  states  will  also  be  constant. 
Assuming  the  error  sources  also  have  constant  autocorrelation 


and  cross  correlation,  the  matrices  R^(i),  R  ^,(i)  and 

e      e£ 

R  (i)  in  (4.21),  (4.28)  and  (4.32)  will  be  constant  for  all 


i.   For  this  case,  let 


Re(i)  =  Qee 


Q 


^1^1 


Q 


^1^2 


Q 


^2^1 


Q 


^2^2 


(4.8la) 


\e'(i)  =  ^ee-  =   ^^e,e'   ^e^e'^ 


(4. Bib) 


R^^(i)  =  Q^^  =   LQp  ^   Qg  J 
ee       ee      e-,  e    ept 


(4.8lc) 


Then  the  matrix  P  whose  entries  appear  in 
(4.48)  can  be  written  as 


F  =  Q   +   E  A^O   +   E  [A^Q^] 
^    1  =  1  '^        1  =  1    ^ 


,-1 


1-1, 


=  Q   +  A[I-A]~^  Q^  +  {A[I-A]~  Q  } 
£  e  ^ 


(4.82) 


by  the  matrix  identity   Z  a"  =  [I-A]"  .   Then 

n=0 
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fT,  =  i-^i— ^-^   (4.83a) 

■^  1  +  a  +  b 


^^63^6/l-^22)-^^12«e2e/^21^6^6/(^22-^lia22+^12^21^Q62e/ 

12  "  " 

^  1  +  a  +  b 


II 


(4.83b)        '\ 

I 


^2 


6/l-^ll)+^12^62e/^2lQ6^6/(^ll-^11^22-^^12^2l'^^6,e2^ 


21 

^  1  +  a  +  b 


(4.83c) 


^^6,e/l-^ll-'^22-^11^22+^12^2l)  ^    ^ag^Q    > 
f   =  2_2 L^(i|.83d) 

'^'^  1  +  a  +  b 


But  F  must  be  a  symmetric  matrix,  which 
implies  that  f.^   ^    ^21  ^°  ^^^^  Equations  (4.83b  and  c)  imply 

that  Q     =  Q     making  Q   a  symmetric  matrix.   This  is 
e-,ep    ^0^1  ^^ 

consistent  with  the  assumption  that  cross  correlations  are 
constant  for  all  i,  since  when  Q^  ^  contains  R„g(i)j  Qg  g 
contains  R„  (i)  =  R  ^(-i)  =  ^^r^^^  •      '^^^^  ^^'^   entries  of  F 

p  Ot  Ot  p  ex  p 


become 


^(l-'^ll-^22-^11^22  +  ^12^21^^e^e^  +  2a^2^^^^^  } 


f   =      ~" i-^ i-^(4.84a) 

11  1  +  a  +  b 

^^2A^e/^l-^11^22  +  ^12^2l)^e^e/^12Qe2e/ 


]  1 
f    =  f 
12     21 


1  +  a  +  b 
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(4.84b) 


^22  "  ^-^    (4.8ilc) 

1  +  a  +  b 


The  entries  of  the  vector  G  (or  G')  which  appear  In  (4.48) 
can  be  written  as 


00 

G  =   E  A^Q    =  [I-A]"^  Q^^  (4.85) 

1=0    ^^  ^^ 


Since  the  object  is  to  apply  (4.84)  and 
(4.85)  to  the  canonic  realizations  of  Chapter  III,  it  can 
be  assumed  the  Q  p=  0  since  for  all  realizations  either 
ep(n)  or  e(n)  is  zero.   Then 


g-L  =  (l-a22)Qe  g/d+a+b)  (4.86a) 


gg  =  B.^^q^   ^/(1+a+b)  (4.86b) 

When  Equations  (4.84)  and  (4.86)  are  applied 
to  the  canonic  realizations,  then  for  the  forms 


SM,TM:     f, ,  =  (l-a-b)Q^  ^  /(1+a+b) 
li  e^e^ 


(4.87a) 


f,^  =  Q    /(1+a+b)  (4.87b) 


12   ^1^1 


f22  =  0  (4.87c) 


Si  =  S2  =  Qe^,/(H-a+b) 
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(4.67d) 


SN,TN:    f^^  =  -^e  e  (^J.88a) 


^TP  =  Qp  p  /(1+a+b)  (H.88b) 


f22  =  0  (4.88c) 


E^  =  0  (4.88d) 


gg  =  Qg  ^/(1+a+b)  (4.88e) 


since  ep(n)  is  zero  for  the  untransposed  forms.   For  the  forms 


Sm'^,Tm'^!   f,T  -  {(l-a-b)Q^  ^  +  2Q^  ^  }/(l+a+b)    (il.89a) 

X  1       X  2 


f,_  =  {-bQ^  ^  +(l-b)Q^  ^  +Q^  p  }/(l+a+b)(4.89b) 
1^       ^1^1       ^1^2    2  2 


f^„  =  {-2bQ  ^  +(l+a-b)Q^  ^  }/(l+a+b)    (4.89c) 
c<-  e -,  e p         e pc p 


g^  =  g2  =  0  (^.89d) 


SN^,TN^:   f,,  =  {-(l+a+b)Q^  ^   +  2Q     }/(l+a+b)   (4.90a) 
11  e^e^^     e^e^ 


f^p  =  {-(l+a+b)Q^  p  +(l-b)Q    +Q    }/(l+a+b) 
12  11        12    2  2 

(4.90b) 


f   =  {-2(l+a+b)Q^  ^^  +(3+a-b)Q    }/(l+a+b) 
22  e^C2         ^2  2 

(4.90c) 
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Si  =  62  =  0  (i|.90d) 

These  expressions  can  now  be  substituted 
into  (i|.^8).   This  yields  for 

ic+e)\  ^  2(c+e)Q^  ^ 

SM,SN,TM,TN:   a/  =  1^  +  L_  +  af  (i).91a) 

^^     (1+a+b)^      (1+a+b)     ^ 


Q   g   +  2Q     +  Q 

SM^,Tm"^:      a.^  =  -^^-^^ i-^ ?-?-        (ii.91b) 

^^         ( 1+a+b  )'^ 

Q 
e  e 

SN*,TN^:      a.^^  =  ?-^  (^.91c) 

^^    ( 1+a+b )^ 


where,  for  the  T  forms,  c  and  e  are  replaced  by  c'  and  e' 
respectively.   These  may  be  written  In  another  form  by 
recalling  that 


Q     =  E[e.(n-l)e,  (n)]  =  E[e  .  (n)e,  (n)  ]      (^.92) 


for  all  1 .   Then  for 


(c+e)e  (n)        p 
SK,SN,TM,TN:   a/^  =  E[{   .^^^^^ +  e(n)}^]       (4.93a) 


„       e^  (n)  +  e,,(n)  ^ 


SN*,TN*:      o/  =  E[{e.{n)/(l+a+b))^]  (1.93c) 

*  AV  <i 
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When  the  filter  is  In  a  limit  cycle  condition, 
it  is  hypothesized  that  a  deterministic  mean-squared  error 
analysis  would  suggest  replacing  Q     by  the  mean-squared 
error  (effective  value).   Since  it  is  well  known  that  the 
limit  cycles  of  the  quantization  error  sources  are  bounded 
by  h/2  for  round  off,  then  for  that  quantization  method  the 
rms  value  of  each  source  must  be  less  than  or  equal  to  h/2. 
Thus  the  rms  value  of  the  composite  errors  e-,  ,  ep  and  e 
would  have  to  be  such  that 


2   <    2  h^ 

^1  -  ^1  T 


'1^2   <   ^1^2  X 


2 

2   ^    2  h^ 

62  1  yg  T 


e   <  V 


T 


(ii.94a) 


^r■  (^.94b) 


(^1.940) 


^  (i^.9^d) 


e^e   <  \i-^v   Tj- 


.2   <  ,2  hi  (H.9^e) 


190 


where  the  bar  indicates  "mean"  and  y.  and  v  are  defined 

after  equations  (A. 55)  and  (4.65a)  respectively.   Replacing 

th®  ^e  e   in  (4.91)  by  the  right  hand  side  of  the  appropriate 

J  1^ 
equation  from  (4.94)  results  In  hypothetical  bounds  on  the 

rms  value  of  the  output  error  as  given  In  Table  4-2.   These 
are  conjectural  and  a  detailed  analysis  Is  suggested  for 
future  research. 

One  example  of  limit  cycles  given  by  Jackson 
[10]  has  been  examined  using  the  formulae  of  Table  4-2.   The 
rms  value  of  the  largest  limit  cycle  In  the  example  was 
4.24h.   The  formula  In  Table  4-2  yielded  a  maximum  rms  value 
of  4.44h.   This  example  Is  offered  not  as  proof  but  rather 
as  motivation. 

D.   CONCLUSIONS 

Tables  4-1  and  4-2  list  formulae  for  the  output  error 
variance  for  two  extreme  cases,  uncorrelated  error  sources 
and  maximum  limit  cycle  rms  values.  It  should  be  pointed 
out  here  that  If  any  of  the  multipliers  In  the  realization 
(not  just  in  the  transfer  function)  are  such  that  it  does 
not  contribute  quantization  errors  then  these  formulae  need 
to  be  modified  (by  reduction  of  \i^   or  v)  to  take  this  into 
account.   For  this  reason,  the  expressions  here  cannot  be 
directly  compared  to  those  previously  presented  in  Table  2-1. 
Another  reason  is  that  previous  analyses  have  not  included 
any  dependence  on  zero  locations.   This  dependence  is  expli- 
cit in  the  expressions  for  the  untransposed  configurations 
since  operations  involving  zeroes  tend  to  follow  those  of 
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Vt  y^   V       (Av)^ 


ARRAY  ^1   ^^2 

SM00'S^00'™0™0  2    0     2      H[l±|±|±g±£]2  h! 


max 

2 


SM,,,SM^,,SN^^,SN^^,TM^  2  0  3  ^^C  c^eH3(l^a.b )  ^2  h! 

SM,,,SM,,,SN,,SN2,,TM,  3  0  2  , 3(c.eH2(  l.a.b )  ^2  ^ 

SM^2  SM21  SN^2  SN21  3  0  3  9['"^:^:r'^'  T 

CM   CM  o  n  h  r2(c+e)  +  ij(l+a+b)-,2  h^ 

CM  h  n  0  rMc+e)  +  2(l+a+b).2  h^ 

SM22,  SN22  ^  0  2  [ jj^j^ ]   X 

SM.J  TM.j  2  2  0  iSEiTiTb-^'  ^ 

SN^J  TN.J  2  2  0  '^CiTiTF^'  T 


TABLE  ^-2      MAXIMUM  MEAN-SQUARED  OUTPUT 
?RROR  FOR  HYPOTHESIZED 
LIMIT  CYCLE  ANALYSIS 
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poles  in  the  flow  between  Input  and  output  In  these  realiza- 
tions, and  thus  are  able  to  have  a  filtering  influence  on 
the  internal  noise  generation.   In  the  transpose  configura- 
tions zero  operations  precede  pole  operations  so  the  zero 
locations  only  influence  the  output  error  by  the  amount  of  ; 
correlation  involved  between  error  sources. 

An  interesting  observation  regarding  the  expressions 
for  error  bounds  by  the  several  investigators  is  the  recur- 
rence of  various  factors.   For  example,  the  factor.  A,  in 
Table  H-1   can  be  written 


A  =  (l-b)[(l+b)^  -  a^]  (^.95a) 


=  (l-b)(l-a+b)(l+a+b)  (^.95b) 

=  (l-b)(l-IaI+b)(H-|a|+b)  (^.95c) 

The  factor  (1-b)  appears  in  the  bounds  of  Sandberg  and 
Kaiser,  Long  and  Trick,  and  Yakowltz  and  Parker  and  in  the 
estimate  of  Jackson.   The  factor  (l-jal+b)  appears  in  every 
bound  of  Table  2-1  for  real  poles  and  in  the  single  bound 
of  Jackson  and  also  of  Bonzanigo.   The  factor  A  appears  as  an 
intermediate  result  in  Parker  and  Hess  [52]  and  Aggarwal  [48], 

Equations  (^1.10)  and  (4.21)  are  the  most  general  result 
of  this  chapter.   They  could  be  very  useful  as  a  tool  for 
understanding  the  interrelation  among  pole-zero  location, 
structure  and  correlation  in  the  accumulation  of  quantization 
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errors  in  the  realizations  of  digital  filters.   Equations 
(^.i<8)  are  the  general  result  for  the  realizations  of  Chapter 
III.   It  is  suggested  that  similar  results  could  be  found 
for  the  ladder  structures  by  formulating  their  algorithms 
via  state  space  techniques.   Another  area  of  promise  is  in 
that  of  multidimensional  filters.   State  space  formulation 
of  these  algorithms  could  possibly  be  expressed  in  terms  of 
tensor  analysis  and  the  expressions  of  (4.10)  and  (4.21) 
likewise . 

Another  area  of  application  for  these  results  is  in  the 
examination  of  realizations  which  perform  quantization  of 
signal  variables  after  additions.   In  this  case  it  is 
suspected  that  accumulated  roundoff  error  will  be  reduced 
since  the  maximum  composite  error  at  time  n  would  be  one 
quantization  step  for  truncation  or  half  a  step  for  roundoff 
rather  than  the  sum  of  the  maximum  error  from  several  sources. 
Quantization  could  also  be  performed  before  multiplications 
allowing  double  precision  signal  variables  to  flow  as  far  as 
possible  through  the  filter  delays.   There  is  often  at  least 
one  path  from  the  input  to  the  output  v;hlch  v;ould  not  require 
quantization. 

The  results  thus  far  still  do  not  indicate  v;hlch  is  the 
"best"  filter  configuration.   The  reason  is  that  the  nature 
of  the  correlation  between  error  sources  has  not  been  completely 
examined.   Extension  of  the  v/ork  of  Appendix  B  is  an  avenue  of 
endeavor  which  v;ould  be  useful  to  pursue.   Once  this  has  been 
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accomplished,  the  "best"  configuration  for  a  given  set  of 
coefficients  could  be  determined  when  the  nature  and 
statistics  of  the  Input  signal  are  known. 

The  results  of  this  chapter  can  also  be  used  In  the 
determination  of  the  optimal  pairing  of  poles  and  zeroes  In 
a  cascade  realization  of  higher  order  filters. 
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Figure  4-1  a.     Realization  of  TMqq  including  quantization  error  sources 


Figure  4-lb.     Realization  of  TmJ  including  composite  error  sources 
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Figure  4-2a.     Non-Linearity  of  Rounding 
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Figure  ^-7}a.     Error  Function  for  Rounding 
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V.   DISCRETE  FOURIER  TRANSFORM  FILTER 

A.   INTRODUCTION 

In  this  chapter  some  relationships  between  the  frequency 
domain  and  time  domain  representations  of  signals  in  finite 
Impulse  response  (FIR)  filters  will  be  examined.   These 
relationships  provide  insight  and  motivation  for  developing 
a  new  convolution  generator  algorithm  which  yields  the  cir- 
cular convolution  of  a  given  sequence  (such  as  a  finite 
Impulse  response)  with  sections  of  data  taken  from  a  continual 
data  stream.   Similarly,  a  new  algorithm  for  performing  the 
Discrete  Fourier  Transform  of  such  sections  of  data  is  pre- 
sented. 


198 


B.   DEFINITION  OP  TERMS 

Recall  from  section  IIA^I  that  the  circular  convolution 

of  two  sequences  of  length  N  was  defined  by  the  sequence 

{y  },  n  =  0,1,..., N-1  such  that 
n 

N-1    ,       N-1 
y  =   I  u  h^    -s  =  ^  u,   ^h  (5.1) 

"   m=0  "^  '^-"^'   m=0  ^^-"^^  "^ 

where  {u  },  {h  }  n  =  0,1,..., N-1  are  the  two  sequences  to  be 
n     n 

convolved.   The  notation  <•>  indicates  modulo  N  arithmetic. 
Subscripts  (or  superscripts)  are  used  as  spatial  or  positional 
identifiers,  whereas  indices  in  parentheses  will  indicate 
time  of  occurrence  or  functional  dependence. 

It  was  also  shown  that  the  same  result  (equation  (5.1)) 
could  be  achieved  via  the  equivalent  operation  of  multiplica- 
tion in  the  frequency  domain  by  defining: 


N-1    , 
U,  =   E  u  W^^  =  DFT[{u  }]   k  =  0,1,...,N-1        (5.2) 
^   n=0  ^  "" 

H,  =   E  h  W^^  =  DFT[{k  }]   k  =  0,1,...,N-1        (5.3) 
K     o  n  n 

n-0 


\  ^   \'\  ^  "  0,1,...,N-1        (5.')) 


To  find  <x>  =  x  modulo  y,  write  x  =  r  +  py  such  that 
P  =  [x/y]  where  [•]  indicates  the  largest  integer  less  than 
or  equal  to  (  • )  .   Then  0  <_  r  <  y  and 
<x>  =  X  modulo  y  =  r  =  X  -  py . 
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whe 


,  N-1 
^n  =  N  JA^    =  IDFTCiYj^}]   n  =  0,1,...,N-1     (5.5) 


re  W  =  exp[-j2TT/N];  (j  =  y^T  )  (5.6) 


Note  that  W"   Is  the  principal  N^^  root  of  1. 

The  subscripts,  k,  above  indicate  the  harmonic  of 
interest,  where  the  fundamental  frequency  is 

il   =  27r/NT  (5.7) 

and  T  is  the  sample  period  in  the  time  domain. 

In  equation  (5-5),  if  n  is  allowed  to  be  outside  the 

interval  [0,N-1]  then  y   =  y<  >,  i.e.,  the  infinite  sequence 
{y  }    is  periodic  in  n  with  period  N. 

C.   THE  DFT  FILTER 

The  term  DFT  FILTER,  as  used  here,  refers  to  a  form  of 
a  finite  impulse  response  (FIR)  filter  presented  by  Gold  and 
Jordan  [53].   This  form  will  be  examined  in  detail  to  gain 
knowledge  about  the  nature  of  the  output,  thus  providing  a 
stepping  stone  to  the  development  of  a  new  convolution 
generator  algorithm. 

1 .   Formulation 

Consider  the  impulse  response,  h(n),  of  an  FIR 
filter,  which  is  zero  outside  the  interval  0  <_  n  <_  N-1.   This 
function  has  a  Z-transform  given  by 


N-1 
H(z)  =   E  h(n)z"^  (5.8) 

n-0 
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which  is  the  transfer  function  of  the  FIR  filter.   Define  the 
sequence,  {h^}  n  =  0,1,..., N-1,  such  that  h   =  h(n).^   Then 
h(n)  may  be  replaced  by  the  Inverse  Discrete  Fourier  Transform 
of  its  Discrete  Fourier  Transform,  (H  } ,  as  defined  earlier. 
Then 

N-1    N-1     ,        .  N-1    N-1 
H(z)  =   I  (^  Z  H^W-"^)z-"  =  I  Z  [H,   Z  (W-^-1)^]   (5.9) 
n=0   k=0  ^  ^,  k=0   ^  n=0 

N-1  N 

Using  the  identity  Z  x  =  r  ~  ^   ,  yields 

n=0     1  -  X 

T    ^-N   N-1     H, 

k=0  1  -  W   z 

-kN 
since  W    =  exp[j2TTk]  =  1. 

Equation  (5.10)  expresses  the  transfer  runclioii  of  the 

FIR  filter  in  terms  of  the  DFT  coefficients,  {H,  },  of  the 

'   k  ' 

sequence  {h  }  =  {h(n)}.   This  is  the  form  utilized  by  Gold 
and  Jordan  to  show  that  "finite  impulse  response"  and  "non- 
recursive"  are  not  synonymous  since,  as  shown  in  Figure  5-1, 
each  term  of  the  summation  in  (5.10)  is  a  recursive  filter; 
yet  H(z)  represents  a  finite  impulse  response  filter. 

But  there  is  more  involved  in  this  form  than  is 
immediately  obvious. 


2 

The  dual  notation  is  used  in  an  effort  to  maintain 

consistency  in  the  subscript/parenthesis  notation.   {h(n)} 

and  {h  }  play  a  dual  role  here  since  (h  }  indicates  coefficient 

position  in  the  filter,  v/hile,  as  an  impulse  response  sequence, 
{h(n)}  is  temporal  in  nature,  yet  they  are  identical  in  an 
FIR  filter. 
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First,  consider  the  terms,  H^,  In  (5.10).   Not  only 


th 


is  Hj^  the  k   harmonic  of  the  DFT  of  {h(n)}  but  it  is  also 
the  value  of  the  transfer  function,  H(z),  evaluated  at  the 
k   N   -root  of  1,  I.e.,  from  (5.3)  and  (5.9)  it  can  be 
seen  that 


Hj^  =  H(z) 


=H(e^^"^) 


"  NT 


-k 
Second,  note  that,  since  W   is,  in  general,  a 

complex  coefficient,  the  terms  of  the  summation  in  (5.10) 

represent  first  order  complex  filters.   Appendix    shows 

that  every  first  order  complex-coefficient  filter  is  also  a 

second  order  real-coefficient  filter.   Furthermore,  since 

|V/~  I  =  1,  the  poles  of  these  filters  lie  on  the  unit  circle 

in  the  z-plane;  that  is,  each  term  represents  a  digital 

resonator  with  frequency 


n^  =  k^  =  ^  (5.12) 


(Cf.  equation  (5.7)).   In  other  words,  the  form  of  the  section 
of  the  filter  in  5-1  between  point  f A  J  and  the  cross-section 
rB^-TB  Vis  that  of  an  oscillator  bank  and  1-z   is  a  comb 
filter.   The  algorithm  represented  by  equation  (5.10)  will 
henceforth  be  referred  to  as  the  DPT  FILTER.   The  motivation 
for  this  phrase  will  become  more  obvious  in  succeeding  sections 
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2.   Response  to  a  Finite  Duration  Input 

It  is  well  known  that  the  response  of  any  filter  to 
an  arbitrary  signal  is  the  linear  convolution  of  that  signal 
with  the  impulse  response  of  the  filter.   The  relationship 
between  the  response  of  an  FIR  filter  to  a  finite  duration 
signal  (of  length  N)  has  an  interesting  connection  to  circular 
convolution.   A  more  complete  connection  is  made  if  one  exam- 
ines the  response  of  the  FIR  filter  to  a  signal  comprised 
of  two  successive  copies  of  this  signal  of  length  N  (total 
length  2N).   Then  a  method  for  finding  the  circular  convolu- 
tion can  be  devised. 

The  response  of  any  filter,  H(z),  to  any  input  signal, 
u(n)  is  given  by  the  linear  convolution: 


y(n)  =  T.      u(m)h(n-m) 
m=-~ 


(5.13) 


When  H(z)  represents  an  FIR  filter  with  an  Impulse 
response  of  length  N,  (h(n);  n  =  0  ,1,  .  .  .  ,N-1),  and  u(n)  Is  zero 
except  for  n  =  0,1,..., N-1,  then  by  dropping  all  terms  In 
(5.13)  known  to  be  zero  (see  Figure  5-2), 


r    n 

I   u(m)h(n-m) 

m=0 

N-1 


0  <  n  <  N-1 


(5.1'0 


y(n)  =  <   I  u(m)h(n-m)    N-1  1  n  <  2N-2 

^  m=n-N+l 


otherwise 
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since  only  on  the  heavy  black  portions  of  the  abscissa  In 

Figures  5-2  d  and  e  are  the  products  in  (5.13)  non-zero.   The 

only  time  (5.1^)  yields  a  term  of  the  circular  convolution 

of  the  two  sequences  {h(m)}  =  {h  }  and  {u(m)}  =  {u  } 

m  m 

m  =  0,1,..., N-1,  is  when  n  =  N-1,  since  this  is  the  only  time 
when  there  are  N  products  of  terms  taken  from  {u(m)}  and  {h(m)} 
in  the  summation.   Then 


N-1 
y(N-l)  =   Z  u(m)h(N-l-m)  (5.15) 

m=0 


That  this  is  a  circular  convolution  term  is  assured  by  the 
fact  that,  in  (5.1^),  because  of  the  limits  on  m,  (n-m)  =  <n-m> 
But  this  is  only  one  term  of  the  circular  convolution. 
To  find  the  other  terms,  it  is  necessary  to  resort  to  another 
technique.   By  passing  two  copies  of  the  data  through  the  FIR 
filter  all  terms  of  the  circular  convolution  are  obtained. 
Since  circular  convolution  can  be  viewed  as  linear 
convolution  if  one  sequence  is  replaced  by  its  periodic 
extension,  consider  the  response  of  the  FIR  filter  to  an 
input  comprised  of  two  successive  copies  of  the  sequence 
{u(m)},  m  =  0,1,...,N-1,  i.e. 

u(n)         0  1  1^  1  ^^-1 
u(n)  =^  u(n-N)       N  £  n  <_  2N-1  (5.16) 

otherwise . 


20^^ 


The  response  to  u(n)  would  be  (see  Figure  5-3) 


0  <  n  <  N-1 


y( 


n)=^ 


r  n 

I ■u(ra)h(n-m) 
m=0  ~   ~ 

n 

2     u(m)h(n-m)    N  <  n  <  2N-1 
m=n-N+l  ~   ~ 

2N-1 

2       u(m)h(n-m)  2N  <  n  <  3N-2 
m=n-2N+l  ~ 


(5.17) 


V. 


otherwise . 


But  in  the  second  summation  the  limits  on  m  imply 
that  0  <  n-m  <  N-1  so  it  can  be  written  immediately  that 


n 


y(n)  =   Z     u(m)h(<n-m>)    N  £  n  <  2N-1 
m=n-N+l 


T  1  O  \ 


Furthermore,  because  of  the  constraints  on  n,  the 
lower  limit  on  m  is  always  less  than  or  equal  to  N,  so  (5.18) 
can  be  broken  into  tv;o  parts,  yielding 


N-1  n 

y(n)  =   E     u(m)h(<n-m>)  +   L  u(m)h(<n-m>)       (5.19) 
m=n-N+l  m=N 


N  £  n  <  2N-1 

But  since  u(m)  =  u(m-N)  for  N  <_  m  <^  2N-1  (from  equation 
(5.16)),  it  follows  that 
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N-1  n 

y(n)  =  l             u(m)h(<n-m>)  +  L   u(m-N)h(<n-m>) 

m=n-N+l  in=N 

N-1  n-N 

=  ^             u(m)h(<n-m>)  +   Z  u(m)h(<n-m-N>) 

m=n-N+l  rn=0 

N-1 

=  I   u(m)h(<n-m>)  N  £  n  <_  2N-1         (5.20) 
m=0 


since  <n-m-N>  =  <n-m>.   The  N  outputs,  y(n),  occurring  for 
N  <_  n  <_  2N-1  are  exactly  the  N  members  of  the  set  of  terms 
resulting  from  the  circular  convolution  of  the  sequences 
{u(m)}  and  {h(m)}  m  =  0,1,..., N-1.   Thus  the  fact  that  circu- 
lar convolution  can  be  viewed  as  a  linear  convolution  if  one 
sequence  is  replaced  by  its  periodic  extension  has  been 
shown  from  a  new  perspective  involving  the  FIR  filter. 

But  when  one  is  processing  continually  received 
data,  it  is  not  efficient  to  stop  and  form  periodic  exten- 
sions of  blocks  of  data  to  achieve  circular  convolutions  on 
that  block. 

Instead,  the  Fast  Fourier  Transform  (FFT)  has  been 
used  to  perform  high-speed  circular  convolution  on  blocks 
of  data  [  15  -  19].   When  these  blocks  are  non-overlapping, 
this  is  a  highly  efficient  method  and  there  is  little  need 
to  overlap  when  the  signal  being  examined  is  highly  stationary 
in  a  statistical  sense.   However,  few  real  processes  are 
highly  stationary.   To  overcome  this,  overlapping  blocks  of 
data  are  processed.   The  question  then  arises,  "How  much 
overlap  does  one  need  and  vvhat  v;ill  it  cost  in  computation  time?" 
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The  following  development  provides  an  algorithm  which  yields 
the  results  in  2N-1  operations  for  complete  overlap  of  data. 
This  development  for  obtaining  the  circular  convolution  of 
blocks  of  data  with  the  impulse  response  of  an  FIR  filter  is 
suggested  by  the  symmetry  property  of  convolution  and  the 
concept  of  periodic  extensions  of  sequences.   In  equation 
(5.16)  the  periodic  extension  of  {u(m)}  was  defined  in  order 
to  find  the  circular  convolution  given  by  (5.20).   Why  not 
use  the  periodic  extension  of  {h(m)}?   Then  the  circular 
convolution  of  {h(m)}  with  {u(m)}  could  be  found  by  the 
linear  convolution  of  one  copy  of  {u(m)}  with  the  periodic 
extension  of  {h(m)}.   Some  very  straight  forward  methods  of 
doing  this  are  possible.   But  with  additional  insight,  a 
method  of  accomplishing  this  with  the  fewest  number  of  summers 
and  a  simpler  topology  is  possible.   This  is  the  object  of 
the  next  two  sections.   The  straightforvrard  methods  will  be 
discussed  after  this  development  for  comparison. 
3 .   State  Analysis 

The  previous  section  states  the  objective  of  finding 
an  algorithm  which  provides  the  circular  convolution  of  a 
sequence  {h(m)}  with  one  block  of  data  from  a  signal  u(n) . 
It  further  suggests  that  some  modification  of  an  FIR  filter 
would  be  suitable  to  this  purpose.   Hov/ever,  in  that  section 
only  finite  length  signals  were  considered.   From  this  point 
on,  the  objective  is  generalized  to  that  of  finding  the  cir- 
cular convolution  of  {h(m)}  with  every  block  of  data  of 
length  N  taken  from  an  arbitrary  continual  data  stream. 
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u(n).   Examining  the  states^  of  the  DFT  FILTER  will  provide 
valuable  Insight  Into  the  nature  of  the  signal  quantities 
Involved. 

Referring  to  Figure  5-1,  define  the  states  of  the 
oscillator  section  as   x^(n),  k  =  0,1,..., N-1  and  let  x(n) 
be  the  signal  quantity  at  point TaJ  ,  i.e. 

x(n)  =  u(n)  -  u(n-N)  (5.21) 

where  u(n)  is  the  input  signal,  which  is  assumed  to  be  zero 
for  n  <  0. 

For  bookkeeping  purposes, define  the  sequence 
{u  (n)},  p  =  0,1,..., N-1,  (where  n  is  now  a  time  index), 
such  that  the  elements  of  the  sequence  are 


u  (n)  =  u(n+p-N+l)       p  =  0  ,1 , . . . ,N-1  (5.22) 


These  sequence  elements  may  be  viewed  as  samples 
taken  from  a  set  of  functions  u  (n)  (see  Figure  5-^a  for 
N  =  i|)  also  defined  by  equation  (5.22).   For  example  consider 
the  sequence,  {u  (n')}  =  {3>  2,  1,  2}  shovm  in  Figure  5-'^b 
plotted  versus  p.   The  values  3,  2,  1  and  2  are  the  func- 
tional values  of  u„(n),  u,  (n),  u^{n)    and  u^(n)  taken  at 

u       1  ^  J 

time  n'  in  Figure  5-^a.   Notice  that  Figure  5-'^b  is  identical 


^States  are  taken  to  be  the  signal  quantites  at  the  inputs 
to  delay  elements  of  the  resonators  of  Figure  5-1. 
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to  the  region  of  the  graph  of  u(n)  (the  top  graph  of  Figure 

5-ila)  Indicated  by  a  heavy  black  line  on  the  abscissa.   This 

heavy  black  line  is  the  "window"  for  viewing  the  block  of 

data  referred  to  as  {u  (n')}.   The  time  index  n'  marks  the 

right  hand  edge  of  the  window.   The  left  hand  edge  of  the 

window  is  then  n'-N+l. 

Next,  define  the  sequence  {s  (n)},  k  =  0,1,..., N-1 

as  the  DFT  of  {u  (n)}.   Note  that  the  time  index,  n,  associates 

P 

these  two  sequences  in  that  {s.  (n)}  is  the  DFT  of  the  block 
of  data  in  the  window  at  time  n,  which  is  by  definition 
{u  (n)}.    Specifically, 

s,  (n)  H   Z  u  (n)wP^  =  I    uCn+p-N+DW^^  (5.23) 

^      p=0  P         p=0 

The  relationship  between  the  states  of  the  DFT  FILTER, 
X,  (n),  and  the  sequence  {Sj^(n)}  can  now  be  found. 
Prom  Figure  5-1,  it  is  seen  that 


Xj^(n)  =  W-^x^(n-l)  +  x(n)  (5.24) 


By  applying  (5.24)  iteratively  (m-1)  times  it  is 
seen  that 

m-1   1 
X,  (n)  =  W-^^x^(n-m)  +  I   W-P^'x(n-p)  (5.25) 

^  ^        p=0 


^It  can  also  be  seen  that  {u  (n)}  is  exactly  the  set  of 
quantities  stored  in  the  M-delay^of  Figure  5-1  at  time  n. 
u  (n)  is  the  quantity  residing  closest  to  but  not  yet  arrived 
°  at  the  summer.   The  quantity  at  the  summer  is  u(n-N)  -  UQ(n-i;. 
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Assuming  zero  initial  conditions,  i.e.  x,  (n)  =  0 
for  n  <_  -1,  then  for  m  =  n+1,  (5.25)  becomes 


n  _ 
X  (n)  =   E  W  P^  x(n-p)  (5.26) 

^      p=0 


Substituting  (5.21)  into  (5.26)  and  assuming  n  >  N-1, 


n         _  V    ^ 
X,  (n)  =   Z  u(n-p)  W  P^  -   I  u(n-p-N)  W~P^         (5.27a) 
^      p=0  p=0 

N-1         ,     n  ,     n 

=   Z  u(n-p)  W'P^  +   Z  u(n-p)  W~P^  -   Z  u(n-p-N)  W~P^ 
p=0  p=N  p=0 

Performing  a  change  of  variables  in  each  of  the  three  summa- 
tions of  (5.27a),  i.e.  q  =  N~p-1,  q  =  p-N  and  q  =  p  respectively, 

x^(n)  =  ''z'u(n.q-N-M)  W^^'^+l^^  .   "z\(n-q-N)  W^^^^^^ 
^      q=0  q=0 

n  _  . 

-   Z  u(n-q-N)  W  ^^  (5.27b) 

q=0 

Combining  the  second  and  third  summations  of  (5.27b),  realizing' 
that  vr^^-'N^^  =  vr^\  yields 

N-1  ,       n  . 

X.  (n)  =  Vr   Z  u(n+p-N+l)  W^*^  -    Z    u(n-p-N)  W"P 
^         p=0  p=n-N+l 

(5.27c) 

The  summation  in  the  first  term  of  (5.27c)  is,  by 
(5.23),  s.  (n) .   The  second  summation  is  zero  by  the  assumption 
that  u(m)  =  0  for  m  <  0.   So 
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Xj^(n)  =  W^  Sj^(n)  (5.28) 


Equation  (5.28)  shows  that  for  any  n  >_  N-1,  the  k 
state  of  the  DFT  FILTER  is  proportional  to  the  k^^  harmonic 
of  the  DFT  of  the  sequence,  {u  (n)},  that  is,  of  the  sequence 
consisting  of  the  present  input  and  the  past  N-1  inputs.  (In 
fact,  this  is  true  for  n  <  N-1  if  one  considers  sequences 
with  leading  zeroes.)   Thus  (5.28)  provides  the  motivation 
behind  the  name  DFT  FILTER  for  the  algorithm  of  equation  (5.10) 
and  Figure  5-1. 

The  state  response  of  the  DFT  FILTER  for  two  specific 
types  of  input  sequence  will  now  be  considered  as  examples 
to  illustrate  the  operation  of  the  comb  filter  section. 

a.   Example  i:   Unit  Pulse  Response 

Let  u(n)  consist  only  of  the  unit  pulse  at  n  =  0. 
This  pulse  is  sent  directly  to  the  resonators  as  x(0)  and 
starts  their  oscillations.   It  is  also  stored  in  the  N-delay 

of  Figure  5-1.   (Figure  5-5  shows  the  result  of  this  input 

-k  n 
for  N  =  4.)   These  oscillations  are  such  that  x^(n)  =  (W   ) 

n  =  0,1, . . . ,N-1. 

From  (5.22),  Uq(H-I)  =  u(0)  =  1  and  Up(N-l)  =  0 

for  p  =  1,2,..., N-1.   So  {u  (N-1)}  is  an  impulse  sequence 

with  the  impulse  at  p  =  0.   On  the  other  hand,  Xj^(N-l)  is 

equal  to  (W"^)^^"^  =  W^  as  expected  from  (5.28)  since  the  DFT 

of  an  impulse  sequence  is  {s^^}  =  (1).   The  factor  W   can  be 
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viewed  as  a  phase  delay  of  2Trk/N  since  V/^  =  exp[-J2Tfk/N] . 
This  phase  delay  is  evident  in  Figure  5-5. 

Recall  that  the  original  pulse  was  stored  in  the 
N-delay  and  now  impinges  on  the  resonators  as  a  negative 
pulse  at  time  n  =  N  since  x(N)  =  u(N)  -  u(0)  =  -1.   But 
Xj^(N)  =  W"^  Xj^(N-l)  +  x(N)  =  0  and  so  for  n   >_  N   the  states 
of  the  DFT  filter  will  be  zero. 

In  effect  then  a  unit  pulse  input  causes  the 
resonators  of  the  oscillator  section  to  oscillate  for  one 
period,  n  =  0,1,..., N-1,  after  which  time  they  are  turned 
off.   Since  an  arbitrary  input  is  slmp].y  a  string  of  weighted 
and  delayed  impulses  and  since  linearity  applies,  the  state 
response  of  the  DFT  FILTER  to  an  arbitrary  input  is  the 
superposition  of  a  string  of  weighted  and  delayed  osclllcitlon 
periods . 

Before  proceeding  with  example  2  some  additional 
insight  into  the  intimate  relationship  between  the  states 
of  the  DFT  FILTER  and  the  DFT  of  sequences  will  be  helpful. 

Consider  equation  (5.23)  which  by  a  change  in  variable 
(p  is  replaced  by  p-1)  can  be  rewritten  as 


s^(n)  =  ?  u  ,(n)  l/P-^)^^  (5.29) 

^      p=l  P-1 


From  (5.22)  it  follows,  in  general,  that 


u  (n)  =  u((n-m)  +  (p+m)-H+l)  =  '^p+m'^^'-^^  (5-30) 


212 


for  any   m.      So    (5.29)    becomes 


Sj^(n)    =  W-^     ^  ^p(^-l)   WP^ 


P=l 

N-1 


=  W-^   {    Z   u^(n-l)    WP^  +   u„(n-l)    W^^   -  uJn-l)W°} 
p  =  0  t^  ^  0 

=  W"^    {s^(n-l)    +   Uj^_^(n)    -   UqCh-I)} 


=  W"^   {Sj^(n-l)    +   u(n)    -  u(n-N)} 


=  W^   {sj^(n-l)    +  x(n)}  (5.31) 


Substituting  (5.31)  into  (5.28)  yields 


Xj^(n)  =  s^(n-l)  +  x(n)  (5.32) 


Equation  (5.32)  says  that  x,  (n)  is  also  equal  to 
the  DFT  of  {u  (n-1)}  plus  an  additional  term.   By  applying 
(5.31)  iteratively  (m-1)  times  it  can  be  shown  that 


s.  (n)  =  W""^^  s,  (n-m)  +  ""z  w"^^^^^^  x(n-q)         (5-33) 
^  ^        q=0 


Substituting  (5-33)  Into  (5.28)  yields 


(         1  M  '^~'\.  , 

X.  (n)  =  W"*-"^"-^^^  s,  (n-m)  +   Z  W"^^  x(n-q)         (5.3^^) 
^  ^        q=0 


213 


So  the  states  of  the  DFT  filter  at  time  n  contain  a  phase 
shifted  version  of  the  DFT  of  previous  sets  of  window  data 
plus  some  additional  terms.   Example  2  will  demonstrate 
this  with  a  periodic  input  signal. 

b.   Example  2:   Response  to  Periodic  Input 

Let  u(n)  be  zero  for  n  <  0  and  periodic  with 
period  N  for  n  >  0  (See  Figure  5-6).   Then  x(n)  (point  (a) 
in  Figure  5-1)  consists  of  only  one  period  of  u(n)  while 
the  window  of  data  stored  in  the  N-delay  after  n  =  N-1 
looks  like  a  continually  progressing  circular  shift  of  that 
period.   The  DFT's  of  such  circularly  shifted  sequences 
is  known  to  be  identical  except  for  a  phase  shift.   Since 
x(n)  =  0  for  n  _>  N  the  resonators  will  continue  to  oscillate 

at  the  rate,  2iTk/NT  radians/second.   This  is  predicted  by 
equation  (5.3^)  when  m  is  not  so  large  that  x(n-q)  is  a 
term  from  the  single  period  of  data  which  enters  the 
oscillator  portion  of  the  DFT  FILTER. 

In  the  next  section  significant  use  will  be  made  of 
the  fact  that  x,  (n)  contains  some  information  about  the 
DFT's  of  any  number  of  sequences  preceeding  {u  (n)}  in  time. 
That  this  information  is  contaminated  by  phase  shifts  and 
additional  terms  will  not  prove  insurmountable. 
^ •   Output  Analysis 

The  relationship  between  the  states  of  the  DFT  FILTER 
and  the  DFT  of  the  input  sequences  provides,  as  shown  below, 
the  knowledge  which  makes  possible  the  extraction  of  complete 
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information  about  the  circular  convolution  of  every  block 
of  data  of  length  N  with  the  impulse  response  of  the  FIR 
filter. 

According  to  Figure  5-1, 


^   N-1 
y^")  =  N  ^^Q^  \^^)  (5.35) 


But  in  the  previous  section  (equation  (5.28)  it  was 
shown  that  x,  (n)  was  proportional  to  s,  (n).   Thus, 

^'°  ^'°  (5.36) 

which  is,  from  equations  (5.5)  and  (5.1)  the  (N-l)th  term 

of  the  circular  convolution  of  the  sequence  {h  }  with  the 

^        n 

sequence  {u  (n)},  since  {s,  (n)}  is  the  DFT  of  {u  (n)}. 
P  '^  P 

So  the  output  may  be  written  as 

N-1  N-1 

y(n)  =  I   u,,,  ,  „^(n)  h(m)  =  E  u^(n)  h(<N-l-m>)   (5.37a,b) 
m=0  <^--L-i"^  ni=0  ^ 


or 


N-1  N-1 

y(n)  =   E  u„  ,   (n)  h(m)  =      I    ujn)    h(N-l-m)       (5.37c,d) 
„  N-l-m  „_n  ^ 

m=0  m-0 


since  <N-l-m>  -  N-l-m  for  the  values  of  m  allowed  in  the 
summations.   These  four  expressions  are  equivalent. 
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However,  If  equation  (5.32)  Is  substituted  into  (5.35), 
then 


-,  N-1  N-1 

y(n)  =  jr  2  H   s  (n-1)  +  i  E  H   x(n)  (5.38) 

k=0  ^  ^        '^  k=0  ^ 


The  first  summation  is  the  zero  term  of  the  circular  convolu- 
tion of  {h  }  with  {u  (n-1)}.   Since  x(n)  does  not  depend  on 
k,  the  second  term  of  (5.38)  is  merely  x(n).h(0)  since 

i  Z  H,  is,  by  definition,  the  zeroeth  term  of  the  IDFT  of 
N   k    ' 

{H  } .   Therefore,  not  only  is  y(n)  the  (N-l)th  term  of  the 

{u  (n)}  convolution,  it  is  also  possible  to  extract  the 
p  ' 

zeroeth  term  of  the  convolution  with  {u  (n-1)}  by  subtracting 
x(n)  h(0)  from  y(n).   In  fact,  N-1  more  convolution  terms 
can  be  extracted  from  y(n),  as  will  be  shown  next. 
Substituting  (5.3^)  into  (5.35)  yields 

y(n)  =  ^  Z  H,  s,  (n-m)  W"^"'-^-^^  +  ^  EH    Z  W  ^^  x(n-q) 
^  k=0  ^   ^  k=0  ^  q=0 

N-1  r      1  \ir        ^~^ 

=  i  Z  H,  s,  (n-m)  W"^'""^''^  +  I    x(n-q)  h(q)   (5-39) 
N  k=0  ^   ^  q=0 

the  first  term  being  the  (m-l)th  term  of  the  {u  (n-m)} 
convolution.   For  notational  purposes  define  y^(n;k)  as  the 
mth  term  of  the  circular  convolution  of  the  sequence 
{u  (k)}  with  {h  },  where  the  index  n  indicates  when  that 
term  is  available  from  y(n).   V/ith  this  notation  then, 
equation  (5.39)  says 
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m-1 

^m-1^'^''^""^^  "  ^^^^    ~      ^   x(n-q)  h(q)  (5.^0) 

q=0 

This  algorithm  is  shown  in  Figure  5-7,  using  the  DFT 
FILTER  form  of  H(z).   Figure  5-8  uses  the  FIR  canonic  form 
of  H(z)  and  utilizes  the  normal  delays  of  that  form  in 
conjunction  with  a  direct  feed-forward  path  to  form 
x(n)  =  u(n)  -  u(n-N) . 

The  algorithm  of  equation  (5.^0)  might  be  called  a 
"circular  convolution  generator."   It  is  most  efficient  when 
one  must  calculate  all  N  convolution  terms  for  every  subse- 

oo 

quence  of  length  N  of  the  "infinite"  sequence  {u(n)}j,.   In 
that  case,  it  requires,  on  the  average,  roughly  two  multi- 
plications per  term  (i.e.,  2N  multiplications  per  convolution) 
instead  of  the  N  multiplications  per  term  required  to  perform 
each  convolution  separately  or  the  (log„  N)  +  1  multiplica- 
tions per  term  required  to  accomplish  each  convolution  via 
the  FFT. 

It  may  be  noted  here  that  it  is  not  necessary  to 
generate  the  term  y   , (n;n-N)  since  this  term  is  available 
as  y(n-N)  =  y^  -,(n-N;n-N).   This  saves  one  delay,  one 
multiplication  and  two  additions  but  is  insignificant  for 
large  N . 

To  illustrate  when  the  terms  of  the  convolution  of 
a  particular  sequence  {u  (m)}  become  available  at  the 
outputs  of  the  circular  convolution  generator  of  Figure  5-7 
or  5-8,  Figure  5-9  shows  the  time  base  of  the  N  outputs. 
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The  points  connected  by  the  diagonal  line  represent  the  N 
terms  of  the  circular  convolution  of  {u  (m)}.   The  last 
output  is  the  redundant  term  y    (n;n-N). 

Section  B  alluded  to  some  straight-forward  methods 
of  performing  this  same  operation.   Figures  5-10  and  5-11 
are  two  such  methods.   The  sequence  (u  (m)}  is  stored  in 
the  first  N  delays  of  these  algorithms  at  time  n  =  m,  there- 
fore these  delay  elements  are  equivalent  to  the  N-delay 
window  positions  of  the  DFT  FILTER.   So  by  the  notation 
convention,  y  (n;k)  represents  the  m   term  of  the  circular 
convolution  of  the  data  in  the  first  N  delays  at  time  k  and 
is  available  at  time  n. 

Figure  5-10  requires  2N-2  delays,  2N-1  multipliers, 
and  N(N-l)  summers  not  Including  the  redundant  operations 
at  the  right  end.   The  worst  thing  about  Figure  5-10  is 
the  horrifying  number  of  circuit  crossings  required. 

Figure  5-11  reduces  the  number  of  summers  and  circuit 
crossings  by  recognizing  that  to  achieve  each  successive 
output  term  moving  from  left  to  right  it  is  only  necessary 
to  add  one  product  and  subtract  another.   Thus  Figure  5-11 
requires  2N-2  delays,  2N-1  multipliers  and  3N-5  summers. 

Again  the  redundant  operations  have  been  removed  from  con- 

2 
slderation.   But  there  are  still  N   -  2N  +3  circuit 

crossings ! 

Figure  5-12  is  a  rearrangement  of  Figure  5-8  v;hich 

5 
has  no  circuit  crossings.   This  arrangement  of  the  circular 

convolution  generator  requires  2N-2  delays,  2N-1  multipliers 
5This  is  particularly  advantageous  in  LSI  technology. 
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and  2N-2  summers.   It  is  believed  that  this  is  the  least 
number  of  operations  required  to  obtain  the  objective. 

Obviously  the  topological  circuit  crossings  cannot  be 
reduced  further. 

Note  that  the  portion  of  Figure  5-12  to  the  left  of 

cross-section  Tc J  -  Tc J  could  be  replaced  by  a  DFT  FILTER 
but  to  no  apparent  advantage  at  this  time. 

D.   MODIFIED  DFT  FILTER 

In  Section  03,  it  was  shown  that  the  states  of  the  DFT 
FILTER  were  proportional  to  the  DFT  of  the  input  sequence, 
{u  (n)}.   It  is  also  possible  to  design  the  oscillator 
section  so  that  the  states  are  exactly  the  DFT  of  {u  (n)}. 
This  will  lead  to  a  minor  refinement  of  the  convolution 
generator.   It  will  also  lead  to  some  observations  that  will 
be  pertinent  in  Section  E  where  a  new  DFT  algorithm  is 
derived. . 

Taking  the  Z-transform  of  equations  (5.2^)  and  (5.28), 


X.  (z)  =  ^^ .  =  W^  S.  (z)   k  =  0,1,...,N-1 

^      1  -  W~^  z~^       '^ 


(5.^1) 


where 


X^(z)  =  Z{x^(n)}  (5.^1a) 


X(z)  =  Z{x(n)}  (5.^llb) 


Sj^(z)  =  Z{s^(n))  (5.'ac) 
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The  right-hand  equality  of  equation  (5.4l)  may  be 
rearranged  to  provide  a  transfer  function  between  X(z)  and 


Sj^(z),  I.e. 


^^^^  1  -  W  ^z  ^ 


The  right  hand  side  of  (5.^2)  represents  a  digital  resonator 
with  a  feed-forward  gain  of  W~   and  a  loop  gain  of  W~  z~  . 
Figure  5-13  shows  a  modified  DFT  FILTER  where  {s  (n)}  are 
the  states  of  the  oscillator  portion. 

Designating  the  output  of  the  modified  DFT  FILTER  as 
v(n),  the  transfer  function  of  this  form  is 

,w  V    ,     -N  N-1  W"^H, 
u^z;     h  k=0  1  -  W  -^z 


In  the  form  of  an  FIR  transfer  function,  (5.^3)  becomes 

N-1 
G(z)  =   E  hCn+Dz""^  (5.^4) 

n=0 

where  h(N)  is  by  convention  equal  to  h(0)  to  be  consistent 
with  the  concept  of  periodic  extensions  of  sequences.   It 
would  also  be  possible  to  replace  h(n+l)  by  h(<n+l>)  which 
is  consistent  with  the  "modulo  N"  nature  of  circular  convolution 


'^This  form  is  related  to  the  Goertzel  algorithm  [  '<  ]  for 
finding  the  DFT  (see  Section  IIA^I  ). 
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Upon  examination  of  the  output  of  this  filter,  it  is 
seen  that 


,  N-1 
v(n)  =  -  z  H  s  (n)  (5.iJ5) 

^^  k=0  ^  ^ 


which  is  the  zeroeth  term  of  the  circular  convolution  of 

{u  (n)}  with  {h  }. 
p  n 

Substituting  equation  (5.33)  into  (5.^15), 


1  N-1  .         T  N-1    m-1   /  ^,  N, 

v(n)  =  ^  S  H,  s  (n-m)  W""^^  +  i  E  H,   E  w"^^"'!^^  x(n-q) 
k=0  ^   ^^  ^  k=0  ^  q  =  0 

m-1 
=  y  (n;n-m)  +  I   x(n-q)  h(q+l)  (5.^6) 

^  q=0 


This  shcv;s  again  that  the  output  contains  sufficient  informa- 
tion to  allow  the  extraction  of  N  convolution  terms  in  a 
similar  manner  as  was  done  previously.   In  this  case 

m-1 
y  (n]n-m)  =  v(n)  -  I   x(n-q)  h(q+l)  (5.^7) 

^  q=0 

This  algorithm  is  shown  in  Figure  5-l4  with  G(z)  repre- 
sented in  the  FIR  form.   G(z)  may  also  be  Implemented,  of 
course,  in  the  modified  DFT  FILTER  form.   In  either  case. 
Figure  5-14  may  be  referred  to  as  the  "modified  circular 
convolution  generator." 

The  miodified  convolution  generator  has  no  particular 
advantage  over  the  one  derived  in  Section  C  except  that  the 
temporal  order  in  which  the  terms  of  a  convolution  associated 
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with  a  particular  sequence  {u  (n)}  become  available  Is  also 

the  ascending  order  of  the  Index,  m,  in  {y  (n;k)}  , 

m  =  0,1,..., N-1.   Furthermore,  this  result  occurs  v/ithout 

relying  on  the  redundant  multiplication  noted  in  Section  C. 

Note  also  that  instead  of  the  apparent  circular  shift  in  the 

index,  m,  there  is,  in  Figure  5-6,  a  circular  shift  of  the 

index  on  the  coefficients,  h  ,  and  that  the  redundant  multi- 

n 

plication  would  be  by  a  second  h„  which  does  not  appear. 

Extending  this  idea,  it  is  then  possible  to  extract  the 

convolution  terms  in  any  circularly  shifted  order  by  a 

suitable  circular  shift  of  the  h  's.   It  will  be  shovm  in 

n 

Section  F  that  by  relying  on  the  symmetry  property  of  con- 
volution it  will  be  possible  to  extract  the  convolution  term 

of  the  u  's.   This  will  have  a  significant  impact  on  a  possible 
application  of  the  circular  convolution  generator. 

E.   A  NEW  DFT  ALGORITHM 

In  the  previous  sections,  the  primary  interest  was  in 
a  time  domain  extraction  of  circular  convolution,  though 
frequency  domain  techniques  were  resorted  to  in  order  to 
gain  an  insight  into  the  properties  of  the  available  signal 
quantities.   In  this  section,  a  new  DFT  algorithm  is  derived 
by  making  use  of  some  of  those  same  insights. 

In  Section  D,  a  transfer  function  between  X(z)  and  S^(z) 
was  derived  which  provided  the  spectrum  of  the  sequence 
{u  (n)}  at  the  output  of  the  oscillator  section  (cross 
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section  (b)  -  (b)  of  Figure  (5-13)).   This  part  of  the 
algorithm  is  not  new.   It  has  been  derived  previously  by 
Dillard  [5^]   as  a  means  of  generating  spectra 
or  spectral  components  of  a  series  of  sequential  sequences 
{u  (n)}.  as  defined  in  Section  C3.   This  algorithm  may  be 
described  as  a  moving  (sliding)  window  DFT  or  uncoupled  DFT. 
As  a  moving  window  DFT,  it  may  be  observed  that  one  may 

feed  one  sequence  of  length  N  into  the  resonators  and  obtain 

2 

the  entire  spectrum  of  that  sequence  in  N  multiplications, 

the  sam.e  number  required  to  perform  the  DFT  in  its  definitive 
form.   However,  the  algorithm  requires  only  N  multiplications 
per  spectrum,  on  the  average,  if  one  is  interested  in  full 
overlap  of  the  moving  window.   Further  savings  are  realized 
if  one  is  interested  in  only  a  few  spectral  components  _,  since 
the  algorithm  provides  each  harmonic  independently  of  all 
others,  i.e.,  it  is  an  uncoupled  form. 

This  algorithm  has  one  major  disadvantage  which  has  not 
been  expounded  or  examined  to  date.   Theoretically  this 
algorithm  provides  the  exact  discrete  spectrum  of  each 
sequence,  (u  (n)},  independent  of  every  other  (u  (m)}.   How- 
ever,  when  finite  precision  arithmetic  is  used,  significant 
errors  destroy  that  idealistic  situation. 

First  of  all,  it  is  apparent  from  studies  of  possible 
pole  locations  in  second-order,  finite  word-length,  real  coef- 
ficient digital  filters  [11   ],  that  no  method  has  been  de- 
vised (or  can  be  devised)  to  place  the  poles  of  a  set  of  such 
filters  exactly  on  the  unit  circle  in  the  Z-plane  and  evenly 
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spaced  in  frequency  (except  for  the  set  {z|z  =  1,  j,  -1,  -j}) 
because  of  the  finite  length  of  the  coefficient  representation, 
This  means  that  there  will  be  errors  due  to  the  fact  that  the 
transfer  function  realized  is  not  the  one  required.   From 
another  point  of  view,  the  problem  is  that  w"^  in  the  defini- 
tion of  the  DFT  is  not  representable ,  in  general,  in  a  finite 
word  length.   So  this  difficulty  applies  to  any  realization 
of  a  DFT  algorithm. 

Secondly,  in  a  finite  word  length  algorithm,  it  is  also 
necessary  to  quantize  the  results  of  multiplication  because 
of  the  extended  word  length  that  operation  implies .   This 
difficulty,  too,  applies  to  any  realization  of  the  DFT. 

But,  the  third  and  most  significant  problem  to  be  faced 
here  is  the  recursive  natur-e  of  the  comb  flltei'  section  of 
Figure  5-13.   In  effect,  it  magnifies  the  problem  of  quanti- 
zation after  multiplication. 

Recursive  digital  filter  sections  have  been  extensively 
studied  in  the  literature  [  39,  51  ]  and  in  Chapter  IV  of 
this  v7ork  with  respect  to  error  generation  and  propagation. 
In  particular,  recursive  filters  are  highly  subject  to  the 
generation  of  limit  cycle  conditions,  especially  when  the 
pole  locations  are  close  to  the  limits  of  stability.   In 
the  case  of  the  DFT  FILTER  the  pole  locations  are  exactly 
on  the  unit  circle,  the  stability  boundary.   It  should  suffice 
here  to  say  that  recursion  should  be  avoided  whenever  possible 
or  practicable . 
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There  Is  a  set  of  complex  coefficients,  however,  which 
do  not  contribute  errors  of  the  type  described  above.   These 
are  the  coefficients  {1,  j,  -1,  - j } .   if  a  transfer  function 
contains  these  and  only  these  coefficients,  there  will  be 
no  shift  in  pole  location  between  the  required  and  the 
realized  filters.   Furthermore,  there  will  be  no  error  intro- 
duced by  quantization  after  multiplication  by  these  coeffi- 
cients and  hence  no  recursive  propagation  of  such  errors, 
even  in  a  recursive  filter.   This  seems  to  be  a  trivial  idea, 
but  is  importance  comes  to  light  in  view  of  an  observation 
by  Hess  [H]  regarding  the  realization  of  digital  resonators 
of  a  desired  frequency. 

Hess  observed  that,  since  the  frequency  of  a  resonator 
is  dependent  upon  the  sample  period,  any  frequency  can  be 
generated.   By  choosing  w  T  equal  to  an  integer  multiple 
of  71/2,  only  multiplications  by  1,  j,  -1  or  -j  are  required 
to  have  an  oscillator  with  frequency  co   as  shown  below. 

It  was  shown  in  Section  D  that  the  DFT  of  windowed  data 
could  be  achieved  through  an  appropriate  arrangement  of 
digital  resonators  v/hose  poles  were  located  at  the  associated 
harmonic  frequencies.   An  algorithm  will  now  be  developed 
which  exploits  the  relationship  between  frequency  and  sample 
period. 

Consider  the  complex  resonator  with  the  transfer  function 


H  (z)  = ^-y  (5.'^8) 

^  1  -  jz 
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H(z)  has  a  pole  at 

z  =  j  =  exp[jTT/2]  (5.^9) 

Equating  the  right  hand  side  of  (5.^9)  with  the  exponential 
form  of  z  on  the  unit  circle: 

z  =  expCjco^T]  (5.50) 

yields  the  resonator  frequency 


w^  =  -^  *  (5.51) 


Furthermore,  one  may  also  oboerve  that  this  pole  is  at 

_N    ■ 
z  =  W  ^  (5.52) 


which  means  that  this  resonator  would  generate  the  DFT 
frequency 


_     _  H$^  _  N    2Tr  (c:    r:o\ 

"n/4  "  "T  ~  it  '  NT  o.jj; 


and  this  is  exactly  co  in  equation  (5-51). 

Now,  if  T  were  to  be  replaced  by  T'  =  2T  then  this  pole 
location  would  correspond  to  half  the  original  frequency. 

However,  it  is  not  part  of  the  approach  at  this  point  to 
consider  changing  the  sampling  rate,  since  it  is  assumed 


226 


that  data  is  being  received  at  a  fixed  sampling  rate.   On 
the  other  hand,  consider  the  complex  resonator 


1  -  jz 


and  evaluate  this  function  on  the  unit  circle.   Thus 


«2(^   ^=  ^^3i:^:2T  =  hrjT-  (5.55) 

1  -  je   ^     1  -  je   ^ 


This  is  exactly  H,(z)  evaluated  on  the  unit  circle  when  T 
is  replaced  by  T'  =  2T.  The  sampling  rate  had  not  been 
changed  in  equation  (5.55)  but  the  effect  is  similar.  In. 
fact,  if  Hp(z)  were  used  in  a  desampling  mode  (v/here  only 
every  other  output  is  considered)  the  effect  would  be  the 
same . 

But  nov;  consider  the  poles  of  Hp(z)  which  occur  at 

,  =  ±^^  =  .le^'^^f   =  eJ^^\  eJ"^  (5.56) 

One  could  achieve  the  effect  of  a  single  pole  at 

z  =  exp[jTT/4]  by  modifying  H^{z)    to  include  a  zero  at 

z  =  exp[j5'T/4] .   Thus 

j5TT/i]      -1        ,        W-5N/8   ^-1 
H  rz)  =  l_i_^r ^_.i_:^w .  (5^,7) 

2  1   -   jz~^  1  -   Jz"'' 
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The  pole,  z  =  exp[jTT/i|]  =  W"^^/^,  Is  one  required  In  any 
DFT  FILTER  algorithm  which  has  N  equal  to  an  Integer 
multiple  of  eight. 

Immediately  obtainable  from  H2(z)  Is  another  pole  required 
in  multlple-of-elght  algorithms;  I.e.,  z  =  W"^^"^-^^.   In  fact 
all  eight  frequencies  for  N  =  8  are  obtainable  from  the  set 
of  transfer  functions: 


1  _  W-5N/8  -1 

%/S^^^    = .  _2     =  ^1^^)  =  "2^^^   ,  (5.58a) 

1  —  J  z 

.    ^-N/8  -1 
G^N/B^z)  =       _2     ""   =  °5^^^  (5.58b) 

-,   ,,-6N/8  -1 

02^/8^2)  =        ,2   ^   =  ^2^^^  (5.58c) 

1  +  z 

,    U-2N/8  -1 

^6N/8(^)  =   :  :  -2   '   =  «6^^)  (5.58d) 

1  +  z 

W-7N/8  -1 

G,„/n(z)  =  ±-^-^ ^-^—  =  G_(z)  (5.58e) 

3N/8        1  +  jz-2        3 

,,-3N/8  -1 

J-  +  J  z 

,,-4N/8  -1 

^8N/8(^)  =        -2   '   =  ^0^^)  (5.58g) 

1  -  z 

JL  ■"  Z 


where  the  subscript  indicates  the  harmonic  number  of  the 
associated  frequency.   That  is,  the  subscript  corresponds 
to  k  in  Figure  5-13. 
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Figure  5-l6  depicts  an  algorithm  which  makes  use  of  this 
development  for  N  =  8.  As  In  Figure  5-13  the  Input  must 
first  pass  through  the  section  (1  -  z~^)  before  reaching 
the  resonators.  Also,  recalling  that  each  resonator  In 
Figure  5-13  had  a  feed  forward  gain  of  W~  ,  each  forward 
path  in  Figure  5-l6  Includes  this  scale  factor  which  was 
not  included  in  equations  (5.58a-f). 

Figure  5-l6  is  a  rather  complicated  algorithm  for  an 
N  =  8  DFT.   It  also  appears  to  require  twice  as  many 
multiplications  as  the  algorithm  of  Figure  5-13.   Some 
further  refinements  would  make  this  a  more  desirable  form. 
Examination  of  equations  (5.58)  yields  these  refinements. 

Consider  equation  (5.58h).   Multiplication  by  W  is 
trivia]-  since  W   -  1.   In  addition,  the  denominator  of 
(5.58h)  is  factorable  into  (1  +  z~  )  (1  -  z~  )  which  means 
(5.58h)  can  be  rewritten  as 


G.(z)  =  — .  (5.59a) 

^      1  +  z"-^ 

In  (5.58g)  a  similar  result  occurs.   Here  W~     =  -1, 


so  that 


G.(z)  =  7"^ ^1  (5.59b) 

U      1  -  z 


In  (5.58c),  W"^^^^  =  j;  thus 


2      (1  +  jz-^)(l  -  jz  ^)    1  +  Jz 
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And  In  (5.58d),  vr^^^^  =  -j ;  so  that 


Gg(z)  =  -J: -^  (5.59d) 

1  -  Jz 

Equations  (5.59a-d)  denote  resonators  with  a  single  delay 
element,  rather  than  the  two  delay  elements  In  equations 
(5.58  c,d,g  and  h) .   The  cancellations  which  were  used  to 
derive  equations  (5.59a-d)  from  (5.58  c,d,g  and  h)  are  not 
possible  in  equations  (5.58  a,b,e  and  f)  without  reintro- 
ducing non-trivial  multiplications  into  the  recursion  loop. 
(A  non-trivial  multiplication  is  one  where  the  coefficient 
is  other  than  1,  j,  -1  or  - j . ) 

Another  simplification  involves  flow  diagram  manipulations 
to  be  performed  on  the  zeros  associated  VJith  the  resonators 

of  equations  (5.58  a,b,e  and  f)  including  the  scale  factor 

-V 

W  '  associated  with  the  appropriate  harmonic. 

Figure  5-17a  shoves  the  zero  paths  associated  with 

-5N/8 
equations  (5.58  a  and  b)  after  observing  that  W  -^    is 

equal  to  -W~^  .  (In  fact,  this  is  true  independent  of  the 

value  of  N,  since 

y-^N/8  ^  exp[(-j2TT/N)(-^N/8)]  =  exp[JTi]  =  -1.) 

(5.60) 
It  is  now  possible  to  remove  the  multiplications  by 
^-N/8  ^^^^^   ^^g  closest  to  the  output  in  Figure  5-17a  by 

V|  /P 

placing  a  multiplicaticn  by  V/"  '    in  each  input  to  the 
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summers  as  shown  in  Figure  5-17b.   Then  one  can  consolidate 
consecutive  multiplications  by  W~^^  to  obtain  a  multipli- 
cation by  W~     which  is  actually  a  multiplication  by  1 
as  shown  in  Figure  5-17c.   The  next  manipulation  is  to 
remove  identical  multiplications  which  occur  in  all  paths 
leaving  a  node  by  placing  a  single  multiplication  before 
that  node.   Figure  5-17d  shows  the  final  arrangement  of  a 
flow  chart  for  equations  (5.58  a  and  b)  which  is  equivalent 
to  that  shown  in  Figure  5-l6 .   (Residual  multiplications  by 
-1  have  been  moved  to  the  right  and  made  part  of  the  summer 
(subtracter)  operation.)   A  similar  manipulation  is  possible 
for  the  zero  paths  of  equations  (5.58  e  and  f ) . 

Figure  5-l8  depicts  an  equivalent  flow  diagram  to  Figure 
5-l6j  incorporating  all  the  above-mentioned  modifications. 
(Note,  only  non-trivial  multiplications  are  drawn  inside  a 
square.   All  trivial  multiplications,  i.e.,  multiplications 
by  1,  j  -1  or  -j  ,  are  shov/n  Inside  circles.)   Three  major 
properties  of  this  figure  should  be  observed. 

First,  all  non-trivial  multiplications  have  been  left 
in  the  form  W~^'^   ,  because  in  this  form  they  are  Independent 
of  N.   On  the  other  hand,  the  subscripts  on  Sj^(n)  are  also 
shown  in  this  manner  to  li:idlcate  their  dependence  on  N.   The 
usefulness  of  this  fact  will  be  seen  later. 

Second,  Figure  5-9  is  a  somewhat  uncoupled  form.   The 
four  single-delay  stages  are  completely  uncoupled  and  the 
two  double-delay  stages  are  palrwlse  uncoupled. 
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Third,  there  are  no  non-trlvlal  multiplications  in  the 
recursive  part  of  the  algorithm.   Therefore,  no  recursive 
error  propagation  will  ensue.   Errors  generated  in  this 
operation  are  only  in  the  output  zero-paths  and  only  effect 
the  DFT  of  one  windovj  of  data. 

A  final  observation  on  this  specific  example  is  that  the 
number  of  non-trivial  multiplications  involved  is  reduced 
from  what  was  required  in  Figure  H.      There  are  six  nontrivial 
multiplications  required  in  Figure  5-13  for  N  =  8,  while 
only  tv.'o  exist  in  Figure  5-l8.   A  first  estimate  on  the 
number  of  multiplications  for  the  FFT  is  N  logpN  =  (8) (3)  = 
2H ,   but  through  manipulations  of  flow  charts  similar  to  that 
performed  on  the  zero  paths;  the  FFT  algorithm  can  also  be 
reduced  to  tv;o  non-trivial  multiplications.   So  nothing  has 
been  gained  in  terms  of  non-trivial  multiplications  over 
the  FFT,  but  some  degree  of  uncoupling  has  been  achieved. 
This  will  shovm  to  be  an  advantage  in  succeeding  paragraphs. 

When  N  =  l6,  four  resonators  (with  four  delays  each)  are 
required  to  provide  the  16  poles  necessary  for  the  transfer 
functions, 

,,-kN/l6 
Wl6<^'  =  ,  _   ,-l.Nyib^-l  '5. 61) 

Again  the  feedback  coefficients  for  the  four  resonators  will 

be  1,  j ,  -1  and  -j . 

It  is  necessary  in  this  case  to  cancel,  three  poles  of 
each  resonator  by  zeroes  to  provide  the  single  pole  of  (5.61). 
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In  the  process  of  doing  so  it  can  be  observed  that  the 
resonators  with  feedback  coefficients  1  and  -1  reduce 
exactly  to  Figure  5-l8  with  N  =  16 .   (Notice  that  N  =  l6 
does  not  change  the  coefficients  but  only  changes  the  sub- 
scripts on  the  outputs,  since  now  W  =  exp[-J2TT/N]  =  exp[-j  2Tr/l6] . ) 
This  reduction  occurs  through  the  same  reasoning  by  which 
Equations  (5.59)  were  derived. 

The  remaining  two  resonators  and  associated  zero  paths 
are  shown  in  Figure  5-19.   When  the  appropriate  flow  diagram 
manipulations  are  performed,  Figure  5-19  reduces  to  Figure 
5-20.   Figures  5-l8  and  5-20  then,  taken  together  form  a 
complete  DFT  algorithm  for  N  =  l6 .   The  outputs  of  Figure 
5-l8  are  the  even  harmonics  and  those  of  5-20  the  odd  ones. 
If  the  algorithm  of  these  two  figures  taken  together  v.'er'=^ 
to  be  built  or  programjned,  there  would  be  available  to  the 
user  DFT  algorithms  for  N  =  1,  2,  ^1,  8  or  l6.   The  only 
parameters  to  be  chosen  are: 

(1)  the  length  of  the  N-delay  at  the  front  end 

(2)  the  number  of  resonator  loops  to  take  advantage  of 

(3)  the  subscripts  of  the  outputs 

But  all  of  these  parameters  depend  on  N  and  can  be  incor- 
porated in  one  switch  or  program  statement  which  indicates 
the  value  of  N.   Notice  also  that  it  is  only  necessary  to 
store  two  non-trivial  complex  coefficients  since 

W-6N/16  ^  ^-3N/8  ^  jw-N/8  (5.62a) 
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T,-9N/l6   _     „-N/l6 

^  -  -W  (5.62b) 


and 


„-2N/l6   ^   „-H/8  ^^^^^^^ 


SO  that   only  W"^^     =   expCjir/^]   and  vr^^^^  =  exp[j37T/4]   need 
be   stored.      These    can   further  be   reduced  to   two   real 
coefficients   since 


R^CW-N/S]  ,  .KJW-3N/83  (5  g^^, 


Another  reduction  of  the  algorithm  Is  possible  if  only 
real  data  is  being  processed.   Then  all  the  resonators  with 
-j  as  the  feedback  coefficient  can  be  removed  since  the 
outputs  associated  with  them  are  available  as  the  complex 
conjugates  of  the  outputs  of  the  resonator  with  the  same 
number  of  delays  and  feedback  coefficient,  j;  e.g. 

where  *  indicates  complex  conjugate.   The  output  ^ni^/i^^^^ 
is  in  Figure  5-20b  while  Sg^j^^g  is  in  Figure  5-20a. 
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Algorithms  for  N  equal  to  a  power  of  2  are  possible 
for  all  higher  powers  as  well.   Figure  5-21  shows  an  eight- 
delay  stage  with  feedback  coefficient,  J,  required  for  N  =  32 
and  higher  powers  of  two.   Figure  5-22  shows  the  equivalent 
stage  after  reduction  of  the  number  of  non-trlvlal  multipli- 
cations.  Figures  5-23  and  5-2^   show  the  sixteen  delay  stage 
required  for  N  =  6^  and  higher  powers  of  two. 

Only  the  stages  with  feedback  coefficient  J  are  shown 

since  the  form  for  the  stage  with  -j  is  the  same  except  that 

-N/32 
all  powers  of  W   -^  in  Figure  5-22  are  replaced  by  the  same 

powers  of  w"^'^'^^^  and  all  powers  of  W~^^^^  in  Figure  5~2k 

-■^N/64 
are  replaced  by  the  same  powers  of  W  -" 

Taking  Figures  5-l8,  5-20,  5-22,  and  5-24  together  as 
one  algorithm,  there  are  available  the  routines  for  finding 
the  DFT  with  N  =  1,  2,  4,  8,  l6,  32,  and  64,  and  the  only 
parameter  that  needs  to  be  changed  is  N. 

Another  advantage  of  this  algorithm  is  that  the  semi- 
uncoupled  nature  of  it  allows  one  to  reduce  the  num.ber  of 
calculations  required  when  only  certain  subsets  of  the  har- 
monics are  required.   For  example,  suppose  one  is  Interested 
only  in  the  odd  harmonics  of  real  data  for  N  =  64.   Then 
only  the  algorithm  of  Figure  5-24  need  be  performed  (inclu- 
ding .le  feedforward  (l-z~   )  path  of  Figure  5-l8)  .   If  only 
every  other  even  harmonic  beginning  with  k  =  2  for  N  =  64 
is  desired,  then  Figure  5-22  yields  all  the  required 
Information,  when  (l-z~   )  precedes  it. 
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If  the  data  being  processed  are  not  from  a  continual 

data  stream,  but  rather  are  Just  N  pieces  of  data,  then  the 

-N 
delay  z    Is  not  necessary.   The  data  can  be  fed  sequentially 

to  the  resonators  as  long  as  the  delay  elements  have  been 
cleared  (set  to  zero  Initial  conditions)  before  entering 
the  first  datum.   The  calculations  on  the  zero  paths  need 
only  be  performed  at  time  n  =  N  (if  it  is  assumed  that  the 
first  datum  is  entered  at  tim.e  n  =  1). 

From  the  previous  paragraph  and  examination  of  the 
resonators  of  Figures  5-l8  through  5-24,  it  becomes  clear 
that  this  method  of  finding  the  DFT  is  essentially  different 
from  other  methods  since  the  first  operations  performed  on 
the  data  involve  adding  all  data,  every  other  datum,  every 
fourth  datum,  etc.,  after  appropi'late  raultiplication  by 
powers  of  1,  j,  -1,  -j .   That  is,  if  this  method  were  performed 
in  parallel  from  the  start  rather  than  with  a  sequential  data 
feed.  Figure  5-25  would  result  for  N  =  8 .   This  also  demon- 
strates that  the  sequential  data  feed  is  easier  to  implement 
in  this  algorithm;  and, with  the  inclusion  of  the  N-delay,  the 
DFT  algorithm  of  Figure  5-l8  allows  complete  overlap  of 
successive  windows  of  data  with  only  7  additions  required 
to  set  up  the  data  rather  than  the  40  additions  of  Figure 
5-25. 
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F.   CONCLUSIONS 

The  convolution  generator  and  modified  convolution 
generator  algorithms  produce  the  circular  convolution  of 
every  block  of  data  in  a  continual  data  stream  with  the 
(finite)  impulse  response  of  an  FIR  filter.   The  algorithms 
are_  simpler  in  the  number  of  operations  to  be  performed  and 
in  topology  than  the  definitive  form  of  generating  circular 
convolutions. 

The  m   impulse  response  of  the  N  outputs  Is  a  copy  of 
the  original  FIR  response  circularly  shifted  m  times  and 
then  delayed  m  sample  times.   Because  of  this  the  algorithm 
could  be  used  to  provide  N  matched  filters  in  one  algorithm, 
where  the  set  of  transmitted  signals  are  the  N  circularly 
shifted  versions  of  a  sequence  of  length  N.   The  maximum 
signal-to-nolse  ratio  for  a  given  transmitted  signal  v/111 
occur  at  the  intersection  of  the  diagonal  line  of  figure 
5-15  and  the  output  line  corresponding  to  the  circular  shift 
of  the  transmitted  signal. 

The  new  DFT  algorithm  presented  here  allows  the  program- 
ming or  wiring  of  a  DFT  of  length  N  =  2^"  to  be  used  for 
performing  the  DFT  of  lengths  a  lower  power  of  two  by  only 
changing  the  parameter  of  length.   The  algorithm  is  designed 
for  sequential  data  feed  which  allows  complete  flexibility 
of  overlap  in  processing  sections  of  data  from  a  continual 
data  stream.   The  zero  path  operations  need  only  be  performed 
when  the  last  datum  of  a  particular  section  of  data  has 
entered  the  resonator  loops.   No  quantization  errors  occur 
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in  the  loops  providing  complete  cancellation  of  previous 
data  not  in  the  section  under  consideration. 
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Convolution  of  a  Length-N  Sequence  with  ths  Single  Period 
Extension  of  another  Length-N  Sequence 
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Figure  5-25.     Parallel   Input  DFT  Algorithm  Equivalent  to  Figure  5-18 
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APPENDIX  A 
CORRELATION  IN  LIMIT  CYCLES 

1.   Introduction 

Limit  cycles  occur  in  digital  filters  as  a  consequence 
of  the  non-linearity  introduced  by  the  requirement  to  quan- 
tize the  results  of  multiplication.   While  analyzing  limit 
cycles,  Parker  and  Hess  [52]  showed  that  the  quantization 
error  sources  exhibit  limit  cycles  which  are  sinusoidal  in 
nature  but  have  a  discontinuity  if  the  principal  magnitude 
is  greater  than  the  maximum  quantization  step  size.   For 
example.  Figure  A-la  shows  a  limit  cycle  where  the  principal 
magnitude  of  the  sinusoid  is  A  while  the  maximum  quantization 
error  is  M.   Figure  A-lb  shows  a  limit  cycle  where  the  princi- 
pal magnitude  of  the  sinusoid  is  B  but  B  is  less  than  M  so 
no  discontinuity  exists. 

Returning  to  Figure  A-la,  notice  that  the  quantization 
error  departs  from  a  pure  sinusoid  at  most  once, always  re- 
turning at  the  next  discontinuity.   Now  if  A  had  been  larger 
than  3M,  the  next  discontinuity  would  have  to  move  the  quan- 
tization error  farther  from  the  pure  sinusoid  as  shown  in 
Figure  A-lc .   In  this  case  the  next  two  discontinuities  make 
the  quantization  error  coincide  vfith  the  original  sinusoid 
again  . 

From  the  previous  discussion,  define  a  limit  cycle  with 
discontinuities  which  cause  the  quantization  error  to  be  at 


2ei\ 


most  once  removed  from  the  sinusoid  (Figure  A-la)  as  a  case 
1  limit  cycle.   If  the  quantization  error  is  twice  removed, 
it  is  a  case  2  limit  cycle.   A  case  k  limit  cycle  is  then  k 
times  removed  from  a  pure  sinusoid;  if  no  discontinuities 
exist,  this  is  case  0. 

Now  consider  two  quantization  error  limit  cycles  in  the 
same  filter.   It  was  shown  by  Parker  and  Hess  that  these 
will  have  the  same  fundamental  frequency  but  one  may  be 
shifted  from  the  other  by  a  phase  6,  as  x,(n)  and  Xp(n)  in 
Figures  (A-la  and  b).   When  considering  two  error  sources  at 
a  time,  define  the  limit  cycle  condition  as  case  k-H   where 
one  error  source  is  in  a  case  k  limit  cycle  and  the  other  is 
in  a  case  I   limit  cycle. 

It  is  thp  object  of  this  appendix  to  show  that  quanti  za- 
tlon  error  sources  are  not  uncorrelated,  especially  In  the 
limit  cycle  condition.   The  correlation  coefficient  for  limit 
cycles  of  the  1-0  case,  and  the  2-0  and  0-0  cases  are  con- 
sidered, assuming  analog  rather  than  digital  functions  for 
ease  of  notation. 

2.   Correlation  In  a  1-0  Limit  Cycle 

Let  the  limit  cycle  functions  for  a  1-0  case  limit  cycle 
be  given  by  the  analog  equivalent  of  Figures  A-la  and  A-lb 
and  let 
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x^(t) 


A  sin  ut       t  e  {0,(})/co}  =  {t  } 

or  t  e  {(TT-(}))/w,(7T+(j))/w)}  -  {t-} 

or  t  e  {(2T7-(j))/a),2TT/co}  =  {t^} 

5 

A  sin  wt  -  2M  t  e  { (})/a) ,  11:1*}  =  {t^} 
^  A  sin  wt  +  2M  t  e  {(tt+4))/co,  (2TT-((>)/a)}  =  {t^^} 


X2(t)  =   B  sin  (tot  +  9) 


where 


{t.}  represents  the  associated  interval 
{-M,M}  is  the  region  of  permissible  values  of  x  (t) 
M  <  A  <_  3M 
0  £  B  <_  M 

(f)  =  sin"^(M/A),  (i.e.  sin~^(l/3)  1  <}>  1  ^/2) 
and  6  is  an  arbitrary  phase  shift. 

The  correlation  between  these  two  functions  is  defined  by 


E[x^(t)x2(t)] 


/   x-^(t)x2(t)f^(t)dt 


'12 


^1^2 


[      /   x^(t)f^(t)dt        /   x|(t)f,^(t)dt] 


1/2 


0 


2Tr/to 
/  x^(t)x2(t)^t 


2tt/w   o  2tt/uj   p  1/2 

[    /  x;(t)^dt      /  x;(t)^t] 

0  -^        '^"        0  -;         -ill 
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where  frp(t)  Is  the  distribution  of  a  random  variable  t, 
uniformly  distributed  over  one  period  of  the  signals. 
Performing  the  integrations  yields 


p^2(^>e)  = 


[tt  -  8r(l-r^)^]cos  9 


[tt^  -  l6TTr(l-r^)^  +  8TTr^cos~-'-(2r^-l)  ]^ 


where  r  =  M/A,  -o  £  r  _<  1 


3.   Correlation  in  2-0  and  0-0  case  Limit  Cycles 

When  the  same  procedure  is  carried  out  for  the  2-0  case 
limit  cycle,  the  resulting  correlation  coefficient  is 


Pl2(^'Q^  " 


1  1 


[TT^+l6TTr^(2TT-sln~^r-3sin~-^3r)-l6TTr{  (1-r^)^ 
1   1 
+(l-9r^)2}]^ 


For  the  0-0  case,  there  is  no  dependence  on  the  ratio  r  =  M/A 
since  then  A  <_  M  and  no  discontinuities  occur  in  x,(t).   In 
this  case  the  correlation  coefficient  is 


p-^^{e)  =   cos 


The  correlation  coefficient  normalized  by  cos  6  is  plotted 
in  Figure  A-2  as  a  function  of  a  =  l/r  =  A/M  for  the  0-0, 
1-0,  and  2-0  cases. 
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APPENDIX  B 
EXPERII^NTS  IN  ERROR  CROSS-CORRELATION 

1.   For  Unquantlzed  Signal  Variable 

In  order  to  gain  an  Insight  Into  the  nature  of  the 
correlation  between  quantization  error  sources  In  digital 
filters,  a  simulation  of  the  process  of  multiplying  a  single 
signal  variable  by  two  different  coefficients  and  rounding 
the  products  was  performed.   A  model  of  the  simulation  is 
shown  in  Figure  B-1,  where  (u)  represents  a  full  precision, 
approximately  uncorrelated  Gaussian  pseudo-random  Input  with 
zero  mean  and  unity  variance. 

The  object  is  to  multiply  the  input  signal  variable  by 
two  coefficients,  A^  and  A^,  and  to  find  the  errors,  e-j^  and 
e        by  rounding  the  products  (indicated  by  the  operation  Q) . 
The  correlation  coefficient  is  computed  by  the  definition 


12   a  a 

^1   2 


where  E[  •  ]  indicates  "expected  value"  and  a^  is  the 
standard  deviation  of  e^  given  by 


for  zero  mean  variables 
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To  understand  what  occurs  in  the  process,  consider  the 
error  functions  given  in  Figure  B-2,  where  it  is  assumed 
that  the  ratio  of  the  coefficients  is 


R  =  ^=|  (B.*) 


and  the  A.  are  both  positive  and  rounding  is  assumed  as  the 
quantization  method. 

The  error  e.  is  given  by 


e^(u)  =  A^u  -  [A^u]   =  A^u  -  k^(u)h  (B.5) 

where  h  is  the  quantization  step  size  and  k^(u)  is  an  Integer 
determined  by 


k,  (u)h    ,  k. (u)h  ,      r    . 

_i LL-  <  u  <  -^ +  -^             (B.6a 

A.      2A,  -   A,    ^  2A, 

11  1  i 


which  implies 


A.u    T  A.u    -,  ,^    r.  ^ 


or 


k^(u)  =  [—  +  ^] 


A.u   T 

-  +  ^j 

I 
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where  [•]-[.  =  greatest  integer  less  than  (•).   It  turns  out 
then  that  the  two  functions  e^(u)  and  e2(u)  must  coincide 
at  Integer  multiples  of  p  =  2h/A^  =  3>h/k^,   since  e^(u)  is 
periodic  with  period  h/A^  and  e2(u)  has  period  h/A2.   These 
periods  coincide  when 

Jh/A^  =  Kh/A2  (B.7a) 

(where  J  and  K  are  integers)  which  implies  that 

J/K  =  A^/A2  =  2/3  (B.7b) 


by  assumption.   So  the  product  e^e^  is  periodic  with  period 


When  ep(u)  is  plotted  against  e, (u),  the  result  is  a 
diagram  like  Figures  B-3(a  through  d)  for  the  ratios  given. 
If  it  is  assumed  that  the  errors  are  uniformly  distributed 
between  ~h/2  and  h/2  and  that  the  joint  probability  density 
is  uniformly  distributed  along  the  solid  lines  of  Figures  B-B, 
it  is  possible  to  calculate  the  correlation  coefficient 
according  to  Equation  (B.l). 

For  example : 

Consider  Figure  B-3c  where  R  is  given  by  (B.4).   The 
joint  probability  density  is  given  by 


•^For  a  continuous  Gaussian  input  with  large  enough 
variance,  this  assumption  is  well  Justified. 


272 


p(e^,e2)    =  p(e2)    *   p(e^|e2) 


(B.8) 


where 


pCe^)   =  i 


(B.8a) 


P(e^  e^) 


|-{6(e^-[|(2e2-2h)])    +   6{e^-l^(2e^-n)V 


+   6(e^-[i(2e2)j)}  ^  <    e.   <   if 


h 
2-2 


|{6(e^-[^(2e2-h)])    +    5 (e^-[i(2e2) ]         (B.8b) 
+   6(e^-[|(2e2+h)])}      -^  <   62  1  ^ 

|{6(e^-[|(2e2)])    +    6 (e^-[|( 2e2+h) ] ) 

"  v ^ -]     L -3  \  "i^ 2  •  "-^J /  ^  /  J       p         "2    ii 


and  6(x)    is   the   Dlrac   delta   function. 

Then 

h         h 

2  2 

E[e,e    ]   =       /  /    e,e  p(e    ,e    )    de   de 

12  j^  ^i^x^ 

"2        "2 

h  h 

?  2 

;    e„p(ep)        /    e   p(c    [62)    de^de2 
h        "^  _h 

"2  2 


h 

2   2e2 


h 

2     e, 


h 

T  e, 


/      ^^2   de,   +      /     -~-  dep  +        / 


de, 


h        ~3h         "        h 
-2  ^ 


2    ■    .-  3-  -^2    ■      ^-  3      -2 


h^ 

IP' 


(B.9a) 
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and 


h 
2  _    .^     2    .      .  >,2 


S  ^  h       ^i^^^i^  ^^i  ^  12  (B.9b) 


2 
so  that 


1 
^12  ~  "l2  (B.IO) 


Performing  the  calculation  for  the  rest  of  Figures  B-3 
yields  for 

R  =  2:   p^2  =  -  f  (B.lla) 


-  =  3:   P^2  "   I         ■  (B.llb) 


R  =  |:   P-L2  =  -  ]^  (B.llc) 


R  =  f :   Pi2  =    13-  (B.lld) 


It  is  suggested  by  Equations  (B.ll)  that  the  rule  for 
finding  the  correlation  coefficient  is  as  follows: 

(1)   Write  A-  and  Ap  as  integer  multiples  of  their 
quantization  step  size,  i.e. 


A^  =  Jq  .  (B.12a) 


Ag  =  Kq  '     .    (B.12b) 


27^^ 


Note  that  q  Is  not  necessarily  equal  to  h. 

(2)   Reduce  J  and  K  until  they  are  relatively  prime 
so  that  the  ratio  of  coefficients  becomes 

R  .  *2  .  K .  m^j  ,      ^ 

where  [m]   means  the  product  of  the  prime  factors  of  m 
which  are  not  prime  factors  of  n. 

(3a)   If  [K]  J  and  [J]  j^  are  both  odd,  then 

=   signum  [R] 
"12   LKjpj  LJJ—  (B.Ha) 

where  signum  [R]  is  +1  for  R  positive,  and  -1  for  R  negative 
(R  =  0  is  a  degenerate  case) . 

(3b)   If  [K]  J  is  odd  and  [J]  „  is  even  or  vice 
versa,  then 


1   signum  [R]  ,    .  s 

^  -nrr m (.B.i'-ib; 


"12  =  -  2  rrr^TTHp, 


The  computer  simulation  verifies  these  results.   There, 
A  and  Ap  were  made  integer  multiples  of  0.01  between  0.01 
and  1.0.   Ten  thousand  random  numbers  were  fed  into  the 

simulation  and  the  products  v/ere  rounded  to  two  decimal 

2 
places.   All  data  for  the  same  ratio  v;ere  averaged  and 


p 

By  the  limits  of  the  experiment,  there  was  more  data  to 

average  for  low  values  of  [J]  j^  and  [K]  j,  thereby  yielding 
better  results.   Averaging  was'"necessary'  due  to  internal 
computer  rounding  v;hich  also  occurs  and  interferes  with 
the  simulated  rounding. 
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plotted  in  Figures  B-i»  through  B-8.   Only  ratios  greater 
than  unity  are  considered  and  the  data  are  divided  into 
groups  where  the  ratios  are  integers  ([J]  „  =  1),  multiples 
of  J  ([J]  TT  =  2  unless  K  is  even),  etc.   (Data  which  appears 
on  a  previous  figure  are  plotted  in  parentheses.) 

In  each  figure,  the  values  of  the  correlation  coefficient 

2 

are  also  shown  multiplied  by  ([J]  y)    .     When  this  is  done 

pK 

the  points  fall  on  the  two  functions  1/R  or  -1/2R.   Thus 


l/R  =  [J  V^^PJ 


([JV  P  -< 


or 


-1/2R  =  -[J]pK/2[K]pj 


(B.15) 


and  therefore 


^/^^W^j 


p=< 


or 


-l/2[J]pK[K]pj 


(B.16) 


verifying  Equations  (B.l^)  except  for  the  factor  signum  [R]. 
But  it  is  obvious  from  Figures  B-2  and  B-3  that  if  R  is 
negative,  A^  and  A^   are  of  opposite  sign  and  the  sign  of  the 
correlation  coefficient  would  be  the  opposite  of  what  it 
would  be  if  A^  and  A^   had  the  same  sign.   (The  experiment  was 
performed  for  A,  and  A^   both  positive.) 
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2.   For  Quantized  Signal  Variable 

In  Section  1,  the  distributions  of  the  errors  were 
assumed  to  be  uniform.   This  is  because  the  input  had  a  full 
precision  (essentially  continuous)  Gaussian  distribution. 
When  the  input  is  quantized  before  the  multiplication  then 
the  distributions  and  joint  distributions  become  discretized 
and  the  results  for  the  correlation  coefficient  are  signifi- 
cantly different.   A  simulation  of  this  case  has  not  been 
rvin  but  the  set  of  points  in  the  (e^  ,e  )  plane  where  the 
joint  error  probability  density  function  has  its  support 
has  been  plotted  for  a  signal  quantization  level  of  h  =  1/16, 
a  multiplier  coefficient  quantization  level  of  q  =  1/8. 
These  are  shown  in  Figures  B-9 .   The  support  is  indicated 
by  a  numeral  in  the  graph. 

The  products  (EJ)h  =  EJ/16  and  (EK)h  =  EK/16  are  the 
errors  corresponding  to  the  multipliers  A  =  Jq  =  J/8  and 
Ap  =  Kq  =  K/8.   If  it  is  assumed  that  every  point  of  support 
has  equal  probability  of  occurrence,  then  the  correlation 
coefficient  can  be  simply  calculated.   By  summing  the  pro- 
ducts of  EJ/16  times  EK/16  for  all  values  of  EJ  and  EK  which 
provide  support  and  dividing  by  the  number  of  points  pro- 
viding support,  E[e  e  ]  is  found.   The  variance  of  each 
error  can  be  found  by  summing  the  squares  of  the  values  of 
EK/16  (or  EJ/16)  for  which  there  is  support  in  that  row  (or 
column)  and  dividing  by  the  number  of  rows  (or  columns)  for 
which  there  is  support.   Note  that  it  is  always  true  that 
if  there  are  n  (>0)  points  of  support  in  one  row  (or  column) 
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then  there  are  n  points  of  support  In  any  other  row  (or 
column)  that  contains  support  (as  long  as  EK  =  8  and  EK  =  -8 
are  considered  the  same  row) . 

No  attempt  has  been  made  to  date  to  calculate  the  corre- 
lation coefficient  for  this  case  or  to  predict  a  formula 
for  its  calculation  based  only  on  knowledge  of  the  relatively 
prime  values  of  the  multiplier  coefficients,  but  this  is 
suggested  for  future  studies  in  this  area. 

3.   Conclusions 

a.  Figures  B-3  show  the  only  possible  values  of  e,  and 
e^   v;hich  can  support  their  joint  probability  density  function 
for  the  ratio  given,  no  matter  v/hat  distribution  the  input 
has . 

b.  Equations  (B.l4)  yield  an  accurate  formula  for  the 
correlation  coefficient  between  two  sources  of  quantization 
error  arising  from  the  same  signal  variable  if  the  distribu- 
tion  of  that  signal  variable  implies  uniform  distribution 

of  the  error  sources. 

0.   Equations  (B.l4)  do  not  reveal  any  dependence  of 
the  correlation  coefficient  on  the  input  quantization  level 
or  the  output  quantization  level.   It  is  hypothesized  that 
these  dependencies  do  exist.   In  addition,  there  may  be  a 
dependence  of  the  quantization  level  of  the  coefficients  and 
the  common  prir:e  factors  of  J  and  K. 

d.   The  patterning  of  the  distribution  support  suggests 
a  similarity  to  the  work  of  Mulcahy  [55].   He  examined  the 
correlation  between  the  input  to  a  multiplier  anc3  the 
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consequent  error  produced  by  quantization.   His  work  could 
be  extended  and  combined  with  the  results  depicted  in 
Figures  (B-9)  to  yield  insight  into  the  nature  of  the 
correlation  between  two  errors. 
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Figure  B-1.     Quantization  error  simulation  model 
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Figure  B-2.     Error  Functions  for     R  = 
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Figure  B-3a.  Error  Distribution  for  R  = 
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Figure  B-3b.     Error  Distribution  for     R 
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Figure  D-3c.     Error  Distribution  for     R 


_     2  _ 
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Figure  B-3d.     Error  Distribution  for     R  =  a^  =  f 
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Figure  B-5.     Correlation  Coefficient  for  Ratios  a  Multiple 
of  One-Half 
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Figure  B-6.     Correlation  Coefficient  for  Ratios  a  Multiple 
of  One -Third 
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Figure  B-9.1a.     Error  Distribution  for  R  =  J/K  =  1 
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Figure  B-9.1b.  Error  Distribution  for  R  =  J/K  =  1 
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Figure  B-9.1c.  Error  Distribution  for  R  =  J/K  =  1 
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Figure  B-9.1d.     Error  Distribution  for  R  =  J/K  =  1 
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Figure  B-9.1e.     Error  Distribution  for  R  =  J/K  =  1 
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Figure  B-9.1f.     Error  Distribution  for  R  =  J/K  =  1 
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Figure  B-9.1g.  Error  Distribution  for  R  =  J/K  =  1 
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Figure  B-9.1h.  Error  Distribution  for  R  =  0/K  =  1 
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Figure  B-9.2.     Error  Distribution  for  R  =  J/K  =  8/7 
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Figure  B-9.3.     Error  Distribution  for  R  =  J/K  =  7/6 
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Figure  B-9.4.     Error  Distribution  for  R  =  J/K  =  6/5 
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Figure  B-9.6a.  Error  Distribution  for  R  ^  J/K  =  4/3 
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Figure  B-9.6b.     Error  Distribution  for  R  =  J/K  =  4/3 
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Figure  B-9.7.     Error  Distribution  for  R  =  0/K  =  7/5 
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Figure  B-9.8a.       Error  Distribution  for  R  =  J/K  =  3/2 
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Figure  B-9.8b.   Error  Distribution  for  R  =  J/K  =  3/2 
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Figure  B-9.9.       Error  Distribution  for  R  =  J/K  =  8/5 
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Figure  B-9.10.     Error  Distribution  for  R  =  J/K  =  5/3 
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Figure  8-9.11.     Error  Distribution  for  R  =  J/K  =  7/4 
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Figure  B-9.12a.     Error  Distribution  for  R  =  J/K  =  2 
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Figure  B-9.12b.  Error  Distribution  for  R  =  J/K  =  2 
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Figure  B-9.12c.     Error  Distribution  for  R  =  J/K  =  2 
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Figure  B-9.12d.  Error  Distribution  for  R  =  J/K  =  2 
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Figure  B-9.13.     Error  Distribution  for  R  =  J/K  ^-  7/3 
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Figure  B-9.14.     Error  Distribution  for  R  =  J/K  =  5/2 
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Figure  B-9.15.     Error  Distribution  for  R  =  J/K  =  8/3 
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Figure  B-9.16a.  Error  Distribution  for  R  =  J/K  =  3 
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Figure  B-9.16b.     Error  Distribution  for  R  =  J/K  =  3 
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Figure  B-9.17.     Error  Distribution  for  R  =  J/K  =  7/2 
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Figure  B-9.18a.     Error  Distribution  for  R  =  J/K  =  4 
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Figure  B-9.18b.  Error  Distribution  for  R  =  J/K  =  4 
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Figure  B-9.19.  Error  Distribution  for  R  =  J/K  =  5 
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Figure  B-9.2D.     Error  Distribution  for  R  =  J/K  =  6 
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Figure  B-9.21.     Error  Distribution  for  R  =  J/K  =  7 
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Figure  B-9.22.     Error  Distribution  for  R  =  J/K  =  8 
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APPENDIX  C 

VERIFICATION  OF  POSITIVITY  OF  FACTORS 
IN  THE  OUTPUT  ERROR  VARIANCE 


1.  Introduction 

In  Chapter  IV  several  expressions  for  the  variance  of 
the  error  at  the  output  of  a  second  order  digital  filter 
caused  by  quantization  noise  are  derived.   Since  a  variance 
Is  always  a  non-negative  number,  these  expressions  should  be 
examined  to  ensure  they  yield  non-negative  results. 

All  the  expressions  Involve  factors  containing  the 
coefficients  of  the  transfer  function  of  the  digital  filter  as 
given  in  Chapter  III.   There  are  conditions  on  the  coefficients 
in  the  characteristic  function  of  the  filter  imposed  by  the 
requirement  that  the  filter  be  stable.   The  characteristic 

function  of  a  second-order  digital  filter  is  given  by 

2 

z  +  az  +  b.   For  stability  it  is  required  that  -1  <  b  <  1 

and  -d+b)  <  a  <  1+b .   There  are  no  restrictions  on  the 
values  which  may  be  taken  on  by  the  coefficients  c  and  e 
(or  c'  and  e')  since  they  do  not  effect  the  stability  of 
the  filter. 

2.  Posltlvlty  of  the  Factor,  A 

The  factor  A,  which  appears  in  every  expression  for  the 
output  error  variance  is  given  by 

A  =  (l-b)[(l+b)''  ~  a^]  (C.l) 
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Since  -1  <  b  <  1  It  Is  clear  that  the  factors  (1-b)  and  (1+b) 
are  both  positive.   The  requirement  that  -(1+b)  <  a  <  (1+b) 
implies  that  |a[  <  | 1+b |  so  that  the  factor  in  square  brackets 
in  (c.l)  is  positive.   Therefore  A  is  always  positive  inside 
the  stable  region. 

In  addition  to  the  positivity  of  A  within  the  stable 
region  of  the  (a,b)  plane,  it  is  obvious  that  A  is  zero  on 
the  boundary  of  that  region,  since  the  boundary  is  defined 
by  the  equations  b  =  1,  a  =  1+b  and  a  =  -(1+b).   Figure  C-1 
shows  the  regions  of  positivity  of  A.   The  points  (a,b)  =  (0,-1) 
and  (0,1/3)  are  critical  points  of  A.   The  point  (a,b)  =  (0,-1) 
is  a  saddle  point  and  (0,1/3)  is  a  local  maximum,  where 
A  =  32/27.   At  the  origin,  A  =  1. 

3.  Positivity  of  the  Factor,  (l+b)/A 

Figure  C-2  shows  a  plot   of  the  factor  (l+b)/A.   It  is 
clear  from  the  previous  section  that  this  factor  is  always 
positive  inside  the  stable  region  since  A  >  0  and  (1+b)  >  0. 
Furthermore  (l+b)/A  has  a  critical  point  at  (a,b)  =  (0,0) 
which  turns  out  to  be  a  local  minimum.   The  value  of  (l+b)/A 
at  the  origin  is  seen  to  be  unity. 

4.  Positivity  of  the  factor,  p  =  [(c  +e  ) (l+b)-2ace]/A 
There  are  several  ways  to  show  that  the  factor, 

p  =  [(c^+e^) (l+b)-2ace]/A  >  0.   First  of  all,  note  that  it 
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Is  only  necessary  to  show  that  the  numerator  is  non-negative 
since,  in  Section  2  it  was  shown  that  A  >  0  Inside  the  stable 
region. 

a.   One  way  to  show  that  p  >  0  is  to  observe  that 


(c^+e^)(l+b)  >  0  (C.2) 


so  that  It  Is  only  necessary  to  show  that  the  largest  value 

2   ? 
of  2ace  is  less  than  (c  +e  )(l+b). 

If  the  product,  ce ,  is  positive  then  the  largest 

2ace  can  be  is  2ce(l+b)  since  a  <  1+b .   Letting  c  =  ye  with 

2 
Y  >  0  forces  ce  =  ye      to  be  positive.   Then 


(c  +e  )(l+b)  -  2ce(l+b)  >  0 


If  and  only  if 

(y^HI)  >  2y  (C.4) 

and  (C.4)  is  always  true. 

For  ce  negative,  2ace  is  maximum  when  a  =  -(1+b). 
Then  to  show  that 


(c^+e^)(l+b)  +  2ce(l+b)  >  0  (C.5) 


let  c  =  -Ye  with  y    >   0 .      Then  (C.5)  is  true  whenever 
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(1+Y^)  >  2y 


as  before. 

It  can  be  noted  here  that  the  numerator  of  p  is  zero 
if  c  =  e  and  a  =  1+b  or  if  c  =  -e  and  a  =  -(1+b)  .   But  then 
A  is  zero  also  and  p  is  a  degenerate  form.   The  numerator 
is  also  zero  for  any  a  and  b  inside  the  stable  region  if 
c  =  e  =  0 . 

b.   Another  way  to  show  that  p  >_  0  is  to  make  use  of 
Section  3  by  writing 


p  =  (c^+e^)[(l+b)/A]  -  2ace/A.  (C.6) 


For  ce  positive 


p  >  (c^+e^-2ce)(l+b)/A  >  c^+e^-2ce  =  (c-e)^  >  0    (C.7) 


since  a  <  (1+b)  and  (1+b) /A  ^  1. 
For  ce  negative 


p  >  (c^+e^+2ce)(l+b)/A  >_   (c+e)^  >_   0  (C.8) 


since  -a  <  (1+b)  . 

c.   A  third  way  to  realize  that  p  >_  0  is  to  observe  that 
p  is  a  quadratic  in  c  or  e  with  the  coefficient  of  the  qua- 
dratic term  being  (1+b) /A  >  1  >  0.   Fixing  e  and  finding  the 
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point  where  the  slope  of  p  with  respect  to  c  Is  zero,  yields 
the  value  of  c  which  make  p  minimum  for  the  fixed  e: 


c*  =  ^  (C.9) 


Using  c*  to  define  the  locus  of  points  where  the 
partial  derivative  of  p  with  respect  to  c  is  zero,  substitute 
c*  into  p  to  form  p*  =  pi   .   This  is  still  a  quadratic  in 
e   opening  upward.   Taking  the  derivative  of  p*  with  respect 
to  e  then  yields  the  value  of  the  parameter  e  which  yields 
the  minimum  of  the  minima 


e*  =  0  (C.IO) 


which  makes 


c*»  =  c*I  „  =  0  (C.ll) 


e« 


so  that  p  is  a  universal  minimum  at  c  =  e  =  0.   Then  p  =  0 
unless  A  =  0.   This  case  will  be  of  no  concern  since  when 
c  =  e  =  0  the  filter  is  degenerate  and  consists  of,  at  most, 
a  single  delay  element.   It  suffices  to  say  that  p  >  0. 


5 .   Nature  of  the  factor,  p+c 
The  factor  p+c  is  given  by 


2   2 
p+c  =  (c  +e  ) (1+b)  -  2ace  +    cA  .^    ^2) 
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This  factor  Is  also  a  quadratic  In  c  or  e  opening  upward  so 
the  same  techniques  as  used  In  Section  4c  can  be  utilized. 
Taking  the  partial  of  p+c  with  respect  to  e  yields 


,*   = 


ac 
1+b 


(C.13) 


for  the  value  of  e  which  makes  p+c  minimum  for  a  fixed  c. 
Substituting  e*  into  p+c  to  form  (p+c)*  =  (p+c)  I  ^   and 
taking  the  derivative  of  (p+c)*  with  respect  to  c  yields 


,*    = 


-A (1+b) 


b2  -  1 


2[(l+b)2  -  a^] 


(C.l^) 


Using  this  value  of  c  in  (C.13)  makes  the  critical 
value  of  e 


,** 


a(b  -  1) 


(C.15) 


Substituting  (C.l4)  and  (C.15)  into  p+c  yields 


^P'*"^Mc*,e** 


_  -(1  -  b^) 


(C.16) 


The  minimum  value  of  (C.l6)  in  the  stable  region  is 
at  b  =  0.   Then 


p+c 


•1/4 


(C.17) 


c=-l/2 
e=-a/2 
b=0 
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Although  the  factor  p+c  is  not  necessarily  positive 
Itself,  the  factors  in  which  it  is  used  in  Chapter  IV  are 
2(p+c)  +  3  and  3p+2c+2  and  by  this  analysis  and  Section  k : 

2(p+c)  +  3  >  2.5  (C.18) 


and 


3P+2C+2  =  2(p+c)+p+2  >  1.5+P  >  1.5  (C.19) 


since  p  >_  0. 

6 .   Other  Factors  of  Interest 

The  precpding  sections  have  been  concerned  with  factors 
which  appear  in  the  expressions  for  the  output  error  variance 

for  the  uncorrelated  case  as  listed  in  Table  4-1.   There  are 

2 
factors  in  the  general  expressions  for  a.      given  by  Equations 

"(4.48)  for  which  knowledge  of  positivity  may  be  useful  in 

future  studies.   These  will  be  discussed  here. 

a.  In  Equations  (4.48a  and  c)  the  factor  multiplying 
f^   is  identical  to  p  in  Section  4  of  this  appendix.   This 
has  been  shown  to  be  non-negative. 

b.  The  factor  multiplying  f  „  in  Equation  (4.48a)  is 


r  -  {c^b^(l+b)-2ce(ab^)+e^[(l+b)-a^(l-b)]}/A      (C.20) 
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This  too  is  a  quadratic  in  c  or  e  opening  upward.   Taking 
the  partial  of  r  with  respect  to  c  yields 


c«  =  3^  (C.21) 


as  the  value  of  c  which  minimizes  r  for  a  fixed  e.   Defining 
r*  =  r|  ^  and  differentiating  r*  with  respect  to  e  yields 


e*  =  0 

so  that  c  =  e  =  0  yields  the  minimum  value  of  r.   The  factor 
r  is  unbounded  on  the  boundary  of  the  stable  region  unless 
b=-lorc=e=0.   Then  it  is  an  indeterminate  form. 

c.  The  factor  1+a+b  multiplying  f   and  f , p  in  Equation 
(^.^8d)  is  definitely  non-negative  by  the  requirement  that 
-d+b)  <  a  £  1+b  in  a  stable  filter. 

d.  The  factor  multiplying  f  _  in  (4.48c)  can  also  be 
shown  to  be  non-negative,  its  minimum  occuring  for  c  =  e  =  0. 

e.  The  factor  that  multiplies  fp^  in  (4.48c)  has  not  been 
tested  completely,  but  it  can  be  shown  that  it  is  a  quadratic 
in  c  or  e  and  that  the  coefficient  multiplying  either  quadratic 
term  is  non-negative.   It  is  expected  that  this  term  is 
non-negative. 

f.  The  factor  (a)  multiplying  f  „  in  Equation  (4.48b) 
can  of  course  be  positive  or  negative. 

g.  The  factor  multiplying  f  ^  in  (4.48a)  can  also  be 
positive  or  negative,  since  the  coefficient  v.'hich  multiplies 
the  quadratic  term  c  or  e  can  be  positive  or  negative. 
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Fig.  C-1.   Sign  of  A  =  (l-b)[(l+b)^-a^]  in  the  (a,b)-plane 
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Fig.  C-2.   Plot  of  (l+b)/A  i 


n 


the  stable  region. 
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APPENDIX  D 

EXPERirCNTS  IN  ERROR  PROPAGATION 
IN  SELECTED  DIGITAL  FILTERS 

1.  Introduction 

Chapter  IV  presents  theoretical  formulae  for  the  predic- 
tion of  the  variance  of  the  output  error  caused  by  quantiza- 
tion of  products  In  a  finite  precision  digital  filter.   These 
formulae  involve  an  Infinite  summation  which  must  be  checked 
for  convergence.   There  Is  no  doubt  that  the  summation  con- 
verges since  It  Is  assumed  that  the  filter  Is  stable  and  the 
entries  of  the  correlation  matrices  are  bounded.   The  question 
Is  whether  In  the  limit  this  summation  yields  the  variance 
of  the  output  error.   But  a  priori  statistics  of  the  errors^ 
are  not  known,  particularly  with  respect  to  correlation. 

In  order  to  gain  some  insight  into  the  processes  Involved, 
computer  simulations  of  a  TM__  filter  v^ere  perform.ed  on  an 
IBM  360.   The  objective  was  to  test  the  convergence  of 
Equation  (4.21)  as  It  applies  to  Equation  (4.il8b)  for  various 
Inputs  and  sets  of  filter  coefficients. 

2.  Experimental  Design 

a.   The  Filter  Simulation 

For  this  simulation,  the  canonic  realization  TMqq 
with  d'  =  0  was  chosen  because  it  does  not  involve  quantiza- 
tion e  '^'ors  In  the  output  equation,  it  has  the  simplest 
exprer,  'on   for  the  output  error  variance,  and  it  does  not 
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require  any  rearrangement  of  coefficients  from  those  given 
In  the  transfer  function.   The  state  equations  and  the  trans- 
fer function  for  this  filter  are  given  by: 


and 


x^(n)  =  -ax^(n-l)  +  c'u(n-l)  (D.la) 


X2(n)  =  -bXj_(n-l)  +  e'u(n-l)  (D.lb) 


v(n)  =  x^(n)  (D.lc) 


H(z)  =   (C  ^_e'z-^)z-^  (j^_2) 

1  +  az   +  bz 


A  subroutine  v/hich  performed  two  algorithms  for  the 
filter  was  designed.   One  algorithm  utilizes  double  precision 
arithm.etic  operations  and  corresponds  to  Figure  3-^2,  the 
other  corresponds  to  Figure  4-la  with  the  errors  being 
generated  by  a  quantization  subroutine  which  rounds  num.bers 
to  one  decimal  place. 

b.   Statistics  Collected 

The  difference  between  the  signal  variables  of  the 
two  filter  algorithms  yields  the  accumulated  errors  Av(n) 
and  y(n).   The  required  statistics  of  these  errors  are 


M  Vi 

o^Chv)    =  I     2  [Av(l)]^  -  [i  I   Av(l)j^      (D.3a) 
i^l  1=1 
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,   M  MM 

Av^y)  =  M  ^  y(i)y^i)  -  Im    2  y(i)][i  E  y(i)]^ 
y    '^  1=1         '^  1=1    '^  1=1 

(D.3b) 


where  M  is  the  number  of  Iterations  performed.   (Writing  the 
statistics  as  being  functionally  dependent  on  Av  and  y  indi- 
cates the  source  of  the  statistic.)   The  value  of  M  for  all 
runs  was  3000. 

The  quantization  errors  were  preserved  and  all  auto- 
and  cross-correlation  functions  (as  defined  in  Appendix  E) 
for  these  errors  were  calculated  up  to  |k|  =  40  I.e. 


1  ^ 
R  _(k)  =  i  E  e  (i-k)e-(i)    -i^O  <  k  <  ^0        (D.k) 
ap      M  ^_,  a      3  —   — 


where  a,  3  =  a,b,c'  and/or  e'. 

Furthermore,  the  quantization  errors  were  summed  as 
appropriate  to  form  the  composite  errors  e, (n)  and  ep(n)  as 
shown  in  Figure  (4-lb)  and  given  by 


e-,(n)  =  e^(n)  +  e^,(n)  (D.5a) 

X  3.  C 

e2(n)  =  e^(n)  +  e^,(n)  (D.6b) 

The  auto-  and  cross-correlations  of  these  errors 

were  also  calculated  as  in  (D.4)  except  UjB  =  1  and/or  2. 

With  these  statistics  available  it  is  possible  to 
form 
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^m^e)  =  P'«(0)  +   ^  A^'r  (i)  +  Z    [A^R  (i)]^   m=0,l,...40 

(D.7) 

where  P  (e)  Is  the  m   partial  sum  of  the  right-hand  side 
m 

of  Equation  (4.21)  and  depends  on  the  errors  e.   (Note  that 
FQ(e)  contains  only  R  (0).)   From  (D.Sb)  it  Is  also  possible 
to  form 


F(y)  =  A^(y)  -  AA  (y)A^  (D.8) 


which  corresponds  to  the  left-hand  side  of  (4.21).   Then 

the  convergence  of  F  (e_)  to  F(y)  can  be  checked. 

2 

For  the  output  error  variance  the  quantities  c    ^{y) 

2 

and  a.  (e)  were  calculated  by  substituting  (D.8)  and  (D.7) 

respectively  into  Equation  (4.48b).   The  convergence  of 

2  2 

a.  (e)  to  o.    (Av)  can  then  be  seen. 
Av  —      Av 

The  auto-  and  cross-correlations  of  the  input  and 
the  states  of  the  low  precision  algorithm  and  of  the  state 
errors  were  also  calculated  and  plotted. 
0.   Filter  Coefficients 

Three  sets  of  filter  coefficients  were  chosen  to 
check  the  effect  of  changes  in  pole/zero  locations  on  the 
error  processes.  These  three  sets  are  given  in  Table  D-la, 
and  the  corresponding  filters  are  labelled  0,  1  and  2  for 
bookkeeping  purposes.  Also  Included  in  Table  D-1  are  the 
resulting  parameters  of  the  infinite  precision  filter  for 
the  given  coefficients  where 
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C   Is  the  damping  coefficient 

w   is  the  undamped  natural  frequency 

YT  is  the  exponent  of  the  envelope  over  the 
unit  pulse  response 

0),   is  the  frequency  of  the  unit  pulse  response 
k.   is  a  phasing  constant  given  by  Equations  (F.12) 
d.   Inputs 

Three  different  input  conditions  were  used  to  derive 
the  filters,  a  zero  mean,  unity  standard  deviation,  essen- 
tially uncorrelated  Gaussian  pseudo-random  one,  and  two  step 
functions,  one  v;ith  magnitude  1.0  and  the  other  0.5-   The 
random  input  was  chosen  to  check  the  shape  of  the  state  and 
state  error  correlations.   The  step  inputs  were  chosen  to 
check  the  response  to  a  highly  correlated  input.   Since  this 
can  also  be  viev;ed  as  a  deterministic  input,  the  output  error 
variance  should  be  interpreted  as  the  mean  squared  error. 

3.   Results  for  Random  Input 

The  results  of  the  simulation  for  the  random  input  are 
shown  in  Table  D-2  where  the  percent  normalised  differences 
are  shown  by 


Lo'^%   =  100[a^^^(e)  -  a^^^(Av)/o^^^(Av)  (D.9) 


Colunui  (11),  headed  by  "o^''^  by  (it.66f)",  indicates 
the  predicted  value  for  the  output  error  variance  given  by 
Equation  (4.66f)  where  it  is  assumed  that  the  error  sources 
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are  uncorrelated  and  uniformly  distributed  so  that 


a^  =  h^/12  =  (0.1)^12  =  0.08333...  (D.IO) 


The  percent  normalized  difference  for  this  theoretical  value 
is  given  in  Column  (12). 

All  these  data  tend  to  confirm  the  convergence  to 
the  correct  value.   The  most  apparent  result  is  that  the 
rate  of  convergence  depends  very  heavily  on  the  value  of 

the  damping  coefficient,  r, .      This  is  shovm  in  Column  (10) 

2 
where  the  value  of  m  is  given  for  the  term  when  a.    (e) 

^  Av  — 

converged  to  and  stayed  within  0.1%  of  the  actual  value. 
Thus  for  filter  2  the  lower  damping  coefficient  required 
that  more  terms  be  summed  to  get  within  0.1^.   It  should  be 
stated  here  that  there  appeared  to  be  some  divergence  be- 
tween term  20  and  term  ^0    for  filter  2.   This  divergence 
was  slow  and  seemed  to  be  peaking  somewhere  near  term  40. 

Another  fact  to  be  inferred  from  Table  D-2  is  that 
the  very  slight  shift  in  zero  locations  between  filters  0 
and  1  resulted  in  a  detectable  change  in  the  output  error 
variance.   This  change  is  more  obvious  for  the  step  input 
to  be  discussed  in  the  next  section.   This  change  is  not 
predicted  by  Column  (11)  since  the  "uncorrelated"  assumption 
yields  formulae  which  do  not  depend  on  c'  and  e'. 

Another  indication  that  the  errors  are  not  uncorre- 
lated is  demonstrated  by  Figures  D-2  through  D-22  which  show 
the  correlation  functions  of  the  states  and  state  errors. 
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(Recalling  that  for  this  filter  the  output  Is  equal  to  state 
X,,  figures  which  Include  this  state  equivalently  include 
the  output  and  figures  which  include  the  state  error  y, 
equivalently  include  Av.)   It  can  be  seen  from  the  autocorre- 
lation of  x-j^  in  Figures  D-2a,~D-3a,  and  D-4a  that  the  conver- 
gence of  the  output  autocorrelation  follows  the  same  pattern 
as  the  convergence  of  the  formula  for  the  output  error  vari- 
ance.  That  is,  in  Figures  D- 2a  and  D- 3a  the  magnitude  of  the 
autocorrelation  of  the  output  enters  and  remains  in  the 
region  ±0.075  at  k  =  ±3,  whereas  in  Figure  D-^a  this  condi- 
tion occurs  at  k  =  ±13.   The  region  ±0.075  is  also  the  region 
in  which  non-zero  values  of  the  autocorrelation  of  the  input 
(shown  in  Figure  D-1)  occur  for  k  7^  0 .   This  is  a  measure 
of  hcvj  close  to  uncorrelated  the  input  is . 

These  Figures  also  show  the  interesting  fact  that 
the  accumulated  state  error  correlations  exhibit  the  same 
pattern  as  the  finite  precision  state  correlations  which, 
by  reference  to  Appendix  F,  can  be  seen  to  exhibit  the  same 
pattern  as  the  theoretical  infinite  precision  correlations. 
This  is  surprising  since  the  quantization  error  sources  and 
the  composite  errors  e-,  and  e^   did  not  exhibit  any  discern- 
ible pattern  in  their  correlations  (and  thus  were  not  inclu- 
ded here).   There  must  then  be  some  pattern  in  the  error 
correlations  which  would  result  in  the  patterns  of  the  state 
error  correlations  v;hen  properly  com.blned.   This  ccm.bination 
might  be  found  by  examining  the  equation 


3^0 


R  (k)  ^  E[y(n-k)y^(n)] 


1=1  J=l 


n-k  n    ,  .  . 

=   Z    Z  a'^'^'^R  (j-l)(A^)"-J  (D.ll) 

1=1  j=l       ® 


There  are  two  more  points  to  be  made  v;lth  respect 
to  the  results  for  the  random  Input  case,  both  Involving  the 
statistics  of  the  quantization  error  sources.   First,  It 
was  observed  that  the  variance  of  all  the  error  sources 
was  very  close  to  h  /12  as  given  in  Equation  (D.IO).   This 
suggests  that  the  errors  probably  were  distributed  in  accor- 
dance with  the  assumption  of  a  uniformly  distribution  between 
-h/2  and  h/2 .   The  mean  of  all  the  sources  was  zero  v;hlch 
also  agrees  with  this  assumption. 

Second,  when  the  correlation  coefficient  for  the 
errors  e  ,  (n)  and  e  ,  (n)  was  exam.ined  it  was  found  that  it 
did  not  agree  with  that  suggested  by  Appendix  B.   This  con- 
firms the  hypothesis  of  that  appendix  since  the  formulae 
given  there  did  not  hold  when  the  input  was  quantized  at 
about  the  same  precision  as  the  filter  variables. 


Notice  that  the  positloni)ig  of  the  multipliers  c'  and 
e'  in  Figure  ^-1  is  the  same  as  that  of  the  two  multipliers 
in  the  experiment  of  Appendix  E. 
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^.   Results  for  Step  Input 

When  the  Input  to  the  filter  was  a  step  function  of 
magnitude  1.0  or  0.5  some  significantly  different  results 
were  achieved.   Table  D-3  lists  these  results  for  mean 
squared  errors  rather  than  variances. 

Column  (11),  headed  by  'hv  (e)  by  (i<.91b)",  indicates 
the  predicted  value  for  the  mean  squared  output  error  given 
by  Equation  (4.91b)  if  the  correlations  are  assumed  to  be 
constant  and  equal  to  the  a  posteriori  value  at  k  =  0 . 

The  percent  normalized  differences  given  are  calculated 
in  the  same  way  as  in  (D. 9). 

Again  the  data  confirm  the  convergence  to  the  correct 
value.   The  rate  of  convergence  also  depends  here  on  the 
damping  coefficient,  C,  but  compared  to  Table  D-2,  more 
terms  are  necessary  because  of  the  highly  correlated  nature 
of  the  errors.   This  is  shown  in  Column  (10)  by  the  value 
of  m  required  for  convergence  to  within  0.1^. 

Notice  that  the  small  shift  in  zero  locations  between 
filters  0  and  1  caused  a  significant  difference  in  the 
resultant  mean  squared  output  error. 

The  plots  of  the  correlations  for  the  step  inputs  were 
nearly  constant  for  the  states,  state  errors,  quantization 
error  sources  and  composite  error  sources.   There  was  a 
tendency  for  these  correlations  to  be  less  than  the  center 
value  at  |k|  =  40,  but  this  deviation  from  an  exactly  constant 
correlation  is  to  be  expected  for  a  real  situation  since  no 
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real  power  spectrum  can  be  the  Ideal  Dlrac  delta  function. 
The  deviation  from  a  constant  value  was  less  than  1.0?  at 
|k|  =  ilO. 

The  mean  squared  error  of  the  error  sources  was  not,  of 

2 
course,  equal  to  h  /12  for  this  case  since  this  is  more  a 

deterministic  case  than  a  probabilistic  one.   Rather,  it  is 

most  likely  that  the  states  were  very  nearly  constant  after 

the  transients  died  out  and  that  the  error  sources  behaved 

similarly.   Thus  the  only  limit  cycle  present  was  a  "sticking" 

effect,  that  is,  a  constant  difference  betv/een  the  "infinite 

precision"  model  and  the  low  precision  model.   The  mean 

squared  error  of  the  error  sources  would  then  depend  upon 

the  value  at  which  the  states  "stick". 

2 

5.   Suggested  Changes  in  the  Experimental  Design 

a.  Filter  Structure 

It  is  recommended  that  filter  algorithms  be  developed 
for  the  rest  of  the  canonic  realizations  of  Chapter  III  in 
order  to  compare  the  results  for  different  realizations. 

b.  Quantization  Subroutine 

Some  error  V7as  introduced  into  the  experiment  by 
having  a  quantization  subroutine  which  attempted  to  round 
to  some  number  of  decimal  points  while  the  program  was  run 
on  a  machine  that  operates  in  binary.   It  is  a  simple  matter 


2 

The  program  used  for  this  experiment  is  available  from 

Dr.  S.R.  Parker,  Electrical  Engineering  Department,  Naval 
Postgraduate  School,  Monterey,  California. 
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to  rewrite  the  subroutine  to  round  to  some  number  of  binary 
points  without  effecting  the  basic  principle  of  the  experi- 
ment.  The  error  experienced  with  the  present  subroutine  is 
probably  in  the  fifth  or  sixth  decimal  place  which  can  be 
significant  for  a  case  like  that  of  filter  2  with  a  unity 
step  input  where  the  mean  squared  error  had  its  first  signi- 
ficant digit  in  the  fifth  decimal  place.   Additionally,  sub- 
routines which  perform  truncation  and  sign-magnitude  trunca- 
tion would  be  useful. 

c.   Statistics  Gathering 

The  calculation  of  variances  in  this  experiment  was 
valid  in  that  no  variances  were  computed  until  the  transients 
of  the  filter  had  decayed  (after  forty  iterations).   But 
the  correlations  calculated  did  make  use  of  errors  generated 
during  the  transient  period.   Although  3000  more  Iterations 
were  performed,  thus  making  the  effects  of  the  transient 
errors  small,  an  even  more  accurate  experiment  would  result 
if  these  errors  were  not  Included. 
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APPENDIX  E 
SPECTRAL  THEORY  IN  DIGITAL  FILTERS 

1.  Introduction 

In  fields  employing  analog  systems,  a  great  deal  of  use 
has  been  made  of  spectral  theory  in  dealing  with  stochastic 
processes.   In  digital  signal  processing,  although  all  the 
necessary  tools  for  using  spectral  theory  are  well-defined 
[^,56], it  has  not  been  relied  upon  very  heavily  in  this  field. 
In  this  appendix,  these  tools  are  presented  and  v/ill  be 
utilized  in  Appendix  F  to  gain  knowledge  about  the  spectral 
properties  of  the  signal  quantities  in  a  digital  filter. 

2.  Synopsis  of  Spectral  Theory  in  the  Z-Domain 
a.   Power  Spectrum  of  a  Single  Random  Process 

Given  a  wide  sense  stationary  random  process,  x(n). 

Its  autocorrelation  function  is  defined  as : 

f 


R  (k)  =  E[x(n-k)x(n)]   k,n  integers         (E.l) 


Since  this  is  a' function  defined  on  a  discrete  domain 
it  is  possible  to  define  its  two-sided  Z-transform  as 

S^(z)  ^  Z[R^(k)]  =   E  R^(k)z"^  (E.2) 


S  {7.)    is  called  the  power  spectrum  of  the  random  process. 
When  normalized  S  (z)  is  called  rhe  power  spectral  density. 
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The  reason  for  this  terminology  Is  seen  when  one  examines 
the  relationship  between  R  (0)  and  the  Integral  of  S  (z) 
around  the  unit  circle  in  the  z-plane. 

Since  R  (k)  is  the  inverse  Z-transform  of  S  (z)  it 
can  be  written  as 


\^^^  =  ^^/^x^^^^"""'^^  (E.3) 


(All  closed  contour  integration  in  this  appendix  is  counter- 
clockwise on  the  unit  circle.) 

Then  the  power  in  x(n)  is  given  by 


E[x=(r,)]  =  R^(0)  =  ^Xs^(.)f 


i   /  S  (eJ"'^)du  (E.I) 

-IT 


which  is  the  mean  square  value  of  the  random  process. 

Other  properties  of  the  autocorrelation  functions 
are  that  it  is  an  even  function  and  that  the  value  at  the 
origin  is  greater  than  or  equal  to  the  magnitude  of  the 
function  for  any  other  value  of  the  argument,  i.e. 


\iO)    >  |R^(k)I  (E.5) 


Properties  of  the  power  spectrum  include: 

(1)   It  is  real  and  non-negative  on  the  unit  circle 

for  real  random  processes.   This  follows  since  R  (k)  is  real 

and  even. 
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(2)   It  is  an  even  function  of  u  on  the  unit  circle 
(z  =  e'^   ).   In  fact.  It  has  even  symmetry  with  respect  to 
Inversion  of  the  argument,  i.e. 


S^(z)  =  S^(l/z)  (E.6) 


This  also  follov/s  from  the  evenness  of  R  (k)  . 

X 

b.   Cross  Power  Spectrum  of  Two  Random  Processes 

Given  two  random  processes  x(n)  and  y(n)  which  are 
mutually  wide  sense  stationary,  their  cross-correlation 
function  is  given  by 


R^^(k)  ^  E[x(n-k)y(n)]  (E.7) 

xy 


The  power  spectrum  of  this  function  is  also  given 
by  Its  Z-transform 


00 

S   (z)  =   2   R   (k)z"^  (E.8) 


The  cross  correlation  function  is  not  necessarily 
even,  but  It  obeys  the  property  that  taking  the  cross  corre- 
lation of  the  random  processes  in  the  opposite  order  has  the 
effect  of  flipping  the  function  about  the  vertical  axis,  l.e 


"xyC^)  ■=  V'-"'  <^-^' 
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The  magnitude  properties  of  the  cross  correlation 


are : 


|R^^(k)|  lkR,(0)  +  R  (0)]  (E.IO) 


xy   '  -  2  X      y 


and 


IR   (k) |2  <  R  (0)R  (0)  (E.ll) 

'  xy    '   —  X    y 


Property  (E.9)  suggests  that  the  cross  power  spectrum 
is  not  necessarily  real  or  even  on  the  unit  circle,  but  that 
same  property  Implies  that 


S   (z)  =  S   (1/z)  (E.12) 

xy       yx 


but 


S   (z)  7^  S   (1/z) 
xy       xy 


necessarily. 

c.   Power  Transfer  In  a  Digital  Filter 
(1)   For  One  Input  and  One  Output 

Let  u(n)  and  x(n)  be  two  random  processes  which 
are  related  by 


x(n)  =  I      u(n-m)h(m)  (E.13) 

m=-~ 
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or 


X(z)  =  H(z)U(z)  (E.14) 

That  is,  u(n)  is  the  input  and  x(n)  the  output 
of  a  digital  filter  with  unit  pulse  response,  h(n),  and 
transfer  function  H(z).   Then  the  autocorrelation  of  x(n) 
is  given  by 


R  (k)  =  E[x(n-k)x(n)] 


CO  00 

=  E{  Z   u(n-k-in)h(m)   Z   u(n-q)h(q)} 
m=-~  q=-co 

00  00 

=   Z     Z    R  (k+m-q)h(m)h(q)  (E.15) 

m=-<»  q=-~   ^ 


and  the  power  spectrum  is 


-k 
S^(z)  =   Z   R^(k)z  "" 


CO       00       oo 

Z     Z     Z    R  (k-q+m)h(m)h(q)z"^ 
k=-~  m=-~  q=-«'   ^ 


=   Z   R  (p)z~^   Z   h(m)z~    Z   h(q)z 
p=_oo         m=-~        q=-«> 


=  S^(z)H(z)H(l/z)  (E.16) 


The  product  H(z)H(l/z)  is  called  the  power  transfer  function 
for  obvious  reasons. 
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(2)   Between  Input  and  Output 

The  cross-correlation  between  the  input  and 
output  of  the  previous  section  is  given  by 


R   (k)  =  E[x(n-k)u(n)] 


00 

E{  Z   u(n-k-m)h(m)u(n)} 


Z   R  (k+m)h(m)  (E.l?) 

in=-oo 


and  the  cross-power  spectrum  is 


OO  CO 

S   (z)  =   E     Z    R  (k+m)h(m)z 
xu      ,  u 


=  Z   R  (p)z"^  Z   h(m)z 
p=_oo         m=-«' 


=  S^(z)H(l/z)  =  S^(l/z)H(l/z)  (E.18) 


When  taken  in  the  opposite  order 


R,^(k)  =   Z   R,,(k-m)h(m)  (E.19) 


^ux^^^  "  S^(z)H(z)  =  S^^(l/z)  (E.20) 


(3)   For  One  Input  and  Two  Outputs 

Let  u(n)  be  a  random  process  used  as  the  input 
to  a  digital  filter  with  two  outputs  (or  equivalent ly,  to 
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two  digital  filters)  and  let  the  two  outputs,  x(n)  and  y(n) 
be  related  to  u(n)  by  the  transfer  functions,  G(z)  and  K(z), 
respectively.   Then  the  cross-correlation  between  x(n)  and 
y(n) 


R   (k)  =  E[x(n-k)y(n)] 


00  09 


=  E{  E   u(n-k--m)g(m)  I      u(n-q)h(q)} 

Ijl=-oo  q  =  -a> 


00        00 


=  Z    E    R  (k+m-q)g(m)h(q) 


and  the  cross  power  spectrum  Is 


00        00        oo 

S   (z)  =   Z     Z     Z    R  (k+m-q)g(m)h(q)z~ 
^      k=-<»  ni=-°°  q=-<» 


=   Z   R  (p)z'"^   Z   g(m)z"'  Z   h(q)z" 
p=_oo         m=-°°       q=-» 


=  S  (z)G(l/z)H(z) 


When  taken  in   the    opposite   order 


00  00 

R_(k)    =      Z  Z        R,(k-m+q)g(m)k(q) 

m=-°°   q  = 


^  ■'^  mrr_oo      r]=_00 


S      (z)    =    S, (z)G(z)H(l/z)    =   S    (l/z)G(z)H(l/z) 


=  ^xy(l/^> 
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APPENDIX  F 
SIGNAL  CORRELATIONS  FOR  A  TMqq  FILTER 

1.  Introduction 

The  computer  simulation  in  Appendix  D  utilized  the  TM^^ 
form  of  a  second  order  digital  filter  as  described  in  Chapter 
III.   The  program  produced  plots  of  the  autocorrelation  and 
cross-correlation  functions  of  various  signal  and  error 
quantities  involved  in  the  processing  of  uncorrelated  and 
correlated  inputs.   In  this  appendix,  the  spectral  theory  of 
Appendix  E  will  be  used  to  derive  the  expected  results  of 
the  plots  had  they  been  performed  for  an  infinite  precision 
filter.   Deviations  from  these  predictions  are  discussed  in 
Appendix  D. 

2.  Unit  Pulse  Responses  in  a  TM-^,  Filter 

Referring  to  Chapter  III,  the  state  equations  for  a  TM-^ 
filter  are 


x^(n)  =  -ax^(n-l)  +  X2(n-1)  +  c'u(n-l)       (F.la) 


X2(n)  =  -bx^(n-l)  +  e'u(n-l)  (F.lb) 


v(n)  =  x^(n)  +  d'u(n-l)  (F.lc) 


Letting  H  (z)  and  Hp(z)  be  the  transfer  functions  from 
the  input  to  x,(n)  and  Xp(n)  respectively  then 
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H^(z) 


[c'  +  e'z  ■^:\z   -^ 

^ ^ 

1  +  az   +  bz 


(F.2a) 


H2(z) 


[e'  +  (ae'-bc')z"-'-]z"-'- 
1  +  az   +  bz 


(F.2b) 


Note  that  for  d'  =  0,  H^(z)  =  H(z),  the  transfer  function 
of  the  filter,  but  for  d'  =  1,  H  (z)  Is  as  given  In  (F.2a) 
but  the  transfer  function  to  the  output  Is 


H(z)  =  z-^  +  ^^'  -^  !i'"^^'"p  =  z"^  + 
1  +  az   +  bz" 


H^(z)    (F.3) 


Now  consider  the  Z-trans forms  of  the  two  functions 


r  e~^"  cos  3nT 


g-,_(n)  =■ 


n  >^  0 
n  <  0 


(F.ila) 


g2(n)  = 


e  "^^   sin  BnT 


n  >_  0 
n  <  0 


(F.i^b) 


which  are 


G^(z) 


1  -  e 


'cos  BTz  ~ 


„  -yT     „„  -1  ^      -2yT  -2 
1  -  2e  '  cos  3Tz    +  e   '  z 


(F.5a) 


G2(z) 


e"^  sin  BTz"  

o  -yT    „„  -1  ^   -2yT  -2 
1  -  2e  '  cos  3Tz   +  e   '  z 


(F.5b) 
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Any  transfer  function  of  the  form 

G(z)  =  -^ — ^ (F.6) 

1  +  az   +  bz 


can  be  written  as 


G(z)  =  c^[G^(z)  +  kG2(z)]z  ^  (F.7) 


by  writing 


,   .    -Iv  -1     c,  (1  +  — z"-'-)z~^ 
(c,+c„z   )z        1'    c. 


G(z)  =  — ^. -^  =  ^ ^  =  c  [G^  (z)+kG  (z)]2"- 

1  +  az+bz      l  +  az-^  +  bz~ 

c  (1  -  e"^'^cos  aTz""^  +  ke"^'^sin  BTz"""")  z"''" 
=  _A fc    o^ 

1  -  2e  '  cos  3Tz   +  e   '  z 
Equating  coefficients  it  Is  found  that 


yT  =  -  I  In  b  (F.9a) 


3T  =  cos~^(-a/2Vb"  )  (F.9b) 


-  +  e~^'^cos  gT       2c2  -  ac-j^ 


k  =  -^-rp =  (F.9c) 

^"^   ^i^  ^T       2c,  (b  -  a2)l/2  VT 

Here  3  is  the  discrete  response  frequency  w  and  y  =  C" 
where  c  Is  the  damping  coefficient  and  co   is  the  undamped 
natural  frequency  of  the  filter. 
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Taking  the  Inverse  Z-transform  of  G(z)  then 


g(n)  =  c^[g^(n-l)  +  kg^Cn-l)] 


r  c^e"^^""-'-^^[cos  6(n-l)T  +  ksln  B(n-1)T]    n  >  1 


L  0 


n  <  0 
(F.IO) 


For  the  transfer  functions  of  Equations  (F.2),  the  unit 
pulse  response  of  the  states  of  the  TM  _  filter  are 


h^(n)  = 


^,g-Y(n-l)Tj-^^g  g(n-i)T  +  k^sin  3(n-l)T]   n  >  1 


n  <_  0 
(F.lla) 


h2(n)  = 


,te-Y^^-^^'^[cos  B(n-1)T  +  k^sin  6(n-l)T]   n  >_  1 


L  0 


n  <_  0 
(F.llb) 


where 


"i- 


2e'  -  ac 


2c' (b  -  a^)l/2V^ 


(F.12a) 


k^  = 


ae'  -  2bc' 


2e'(b  -  a^)^/^  W 


(F.12b) 


and  Y  and  B  are  given  by  (F.9a  and  b) 
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3.   State  Response  to  an  Uncorrelated  Input 

When  the  input,  u(n) ,  to  a  digital  filter  is  a  wide 
sense  stationary  uncorrelated  random  process  its  autocorrela- 
tion function  is  given  by 


R^(k)  =  E[u(n-k)u(n)]  = 


N, 


k  =  0 


k  ?^  0 


(F.13) 


where  N„  is  the  mean  square  power  in  u(n) .   Its  power 
spectrum  is  then 


S  (z)  =  Z[R  (k)]  =   Z   R  (k)z"^  =  N„ 
u       "■  u       ,   ^  u  0 


(F.lil) 


Re 


ferring  to  Appendix  E,  the  power  spectra  for  x,(n) 


and  Xp(n),  the  states  of  the  TM  _  filter,  are 


^x  ^^^  "  S^(z)H^(z)H^(l/z)  =  NqH^(z)H^(1/z) 


(F.15) 


^x  X  ^^^  "  S^(z)H^(l/z)H2(z)  =  NqH^(1/z)H2(z)      (F.16) 


\x^^^)   =  S^(z)H.(z)  =  NqH.(z) 


(F.17) 


For  d'  =  0,  the  output  povjer  spectrum  is  given  by  (F.15) 
with  1=1  since  then  v(n)  =  x, (n) .   This  is  also  the  cross 
spectrum  between  the  output  and  x-,  (n)  .   The  cross  spectrum 
between  the  input;  and  output  is  (F.17)  with  i  -  1  and  the 
cross  spectrum  between  the  output  and  XpCn)  is  given  by  (F.l6) 
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The  auto-  and  cross-correlations  involved  are  by  defini- 
tion the  inverse  Z-transform  of  the  spectra.   But  observe 
that  each  of  the  spectra  is  the  product  of  two  functions  in 
the  z-domain.   By  the  convolution  theorem  this  corresponds 
to  the  convolution  of  the  inverse  Z-transforms  of  the  two 
functions  being  multiplied.   If  the  inverse  Z-transform  of 
H.(z)  is  h.(k),  then  the  inverse  Z-transform  of  H. (1/z)  is 
h. (-k)  so  that 


iR,,(k)  =  Z"^[H.(z)H.(l/z)]  =  h,(k)*h,(-k) 
0  -"-J  -'■      J 


=  E   h. (m)h.(-(k-m))  =  Z  h.(m)h.(m-k) 
jn=-oo  -'■    J  m=-"  ■"■    "^ 

(F.18) 

which  is  the  correlation  of  h.(k)  with  h.(k).   Since  Equations 
(P. 11)  require  that  h.(k)  be  zero  for  k  <_  0,  (F.l8)  can  be 
written  as 


rf-R.  .(k) 


Z  h. (m)h.(m-k) 
m=l  ^    ^ 


I        h.(m)h.(m-k) 
1    J 


m=k+l 


k  <  0 


k  >  0 


L  h.  (m+l)h.  (m-k+1)    k  <_  0 
m=0  ^      J 


E  h  (m+l)h. (m-k+1)    k  >  0 
m=k  ^      '^ 


(F.19) 
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Considering  the  functions  of  Equations  (F.ll),  then 


N, 


h^(in+l)h  (m-k+1)  =  K^^  e"^^^"^"^^'^[cos  BmT  +  k^sln  BmT] 


•[cos  6(m-k:)T  +  k.sln  6(m-k)T] 


(F.20) 


since  the  Indices  m+1  and  m-k+1  are  at  least  unity  for  the 
values  of  m  and  k  in  (F.19),  and  K  .  is  a  constant  involving 
N_  and  the  scaling  constants  c'  and  e'  in  (F.ll). 

By  a  change  of  variable  in  the  second  sununatlon  of  (F.19) 
the  correlation  function  R. .(k)  becomes 


R.  .(k)  =. 


K 


It     ^^ 

l.J  r\  — 


m=0 


[cos  3(m-k)T  +  k.sln  B(in-k)T] 

J 


K 


.  .e"^!'^''^  Z  e~^^"^^[cos  B(m+k)T  +  k^sin  B(m+k)T] 


ni=0 


[cos  3mT  +  k.sln  3mT] 

ti 


k  >  0 
(F.21) 


It  can  be  shown  by  the  use  of  trigonometric  identities  that 


R.  .(k)  =  K.  .e  ^'^'"^{(J-'.-cos  BkT  +ij;..sln  glk|T}  (F.22) 
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where 


,  (l-k,k.)(l-e  ^^'^cos2BT)  +  (k,+k,)e"^^'^sin2BT  l+k,k, 

^     =  £r— i_J .  -  J +  ^  J  -I 

*iJ   ^  l-2e-^^'^'cos26T-.e-^^^  ^l-e"^^^^ 

(F.23) 

,  (k,+k,)(l-e"^^'^cos2BT)-(l-k,k.)e"^^'^sin2BT  k,-k. 

(F.2i4) 


using  the  plus  sign  for  k  >_  0  and  the  minus  for  k  _<  0 .   Notice 

that  for  1  =  J ,  R. .(k)  Is  an  even  function  of  k  and  its  maxi- 
"^ '   ij 

mum  occurs  at  k  =  0,  so  that  R. .(k)  is  a  valid  auto-correlation 
function. 

The  point  to  be  made  here  is  that  R. ^(k)  is  bounded  in 


tude  by  k^  .  ( (j)^j+ 4^?^)  ^^^  e  '^i^l'^.   For  1  =  j,  this  bound 
is  unambiguous.   But  for  i  7^  j  the  choice  of  the  sign  in  the 


magni 

-  J 


expression  for  4^. -must  be  the  one  which  maximizes  ^-s  ^- •  None- 
theless,  this  bound  has  the  same  form  for  every  correlation 
function  involving  the  states  and  output  of  the  Infinite 
precision  TM  ^  filter  with  d'  =  0  when  the  input  is  an 
uncorrelated  wide  sense  stationary  random  process .   A  similar 
analysis  would  show  that  this  is  true  for  any  filter  realiza- 
tion with  the  same  characteristic  function.   The  only  excep- 
tion is  that  at  the  oi'igin,  R„(k)  ,  the  output  autocorrelation 
will  have  an  additional  constant  term  when  d  (or  d')  =  1. 
This  occurs  because  d  =  1  Implies  a  transfer  function  of 
unity  between  the  input  and  output  in  parallel  with  the 
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rest  of  the  transfer  function.   Then  the  input  autocorrela- 
tion pulse  at  k  =  0  Is  transferred  directly  to  the  output. 
To  show  this,  let 


X{|}=H(z)  =  1  -.  G(z) 


(F.25) 


where  G(z)  Is  In  the  form  of  (F.6).   Then 


and 


H(z)H(l/z)  =  1  +  G(z)  +  G(l/z)  +  G(z)G(l/z)   (F.26) 


N 
R^(k)  =  ^(5H(z)H(l/z)dz/z 


(F.27) 


for  an  uncorrelated  input  so  that 


o 


1  +  2g(0)  +   Z  g'^Cm) 
m=0 

oo 

g(-k)  +   Z  g(m)g(m-k) 
m=0 

00 

g(k)  +   L  g(m)g(m-k) 
m=k 


k  =  0 

k  <  0   (F.28) 

k  >  0 


since    g(n)    =   0    for  n   <    0.      So   for  k   7^   0   R    (k)    is   bounded 


V 


-Y I k I T 
by  a  constant  times  e  ' '  '   but  at  the  origin  there  is  an 


additional  [1  +  g(0)]N   term. 

o 
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4 .   state  Response  to  a  Highly  Correlated  Input 

In  the  previous  section  an  uncorrelated  Input  was  used 
to  determine  the  correlation  functions  of  the  states  of  the 
filter.   The  Input  power  spectrum  In  that  case  was  found  to 
be  a  constant.   At  the  other  extreme,  suppose  the  auto-corre- 
lation of  the  Input  is  a  constant.   This  Is  an  extreme  case 
because  as  noted  in  Appendix  E  the  magnitude  of  the  auto- 
correlation function  at  the  origin  is  greater  than  or  equal 
to  the  magnitude  at  any  other  value  of  the  argument.   It 
will  be  shown  here  that  this  kind  of  input  yields  correlation 
functions  of  the  states  which  are  also  constant. 

a.   The  Power  Spectrum  for  a  Constant  Autocorrelation 
The  first  problem  involved  in  finding  the  power 
spectrum  of  a  constant  autocorrelation  function  is  that  a 
constant  discrete  function  is  not  absolutely  summable  so  that 
the  application  of  the  two-sided  Z-transform  may  not  be 
valid.   This  difficulty  can  be  removed  by  letting  the 
constant  autocorrelation  be  given  by 


R  (k)  =  11m  e"'^'^'       a  >  0  (F.29) 


It  is  now  possible  to  apply  the  two-sided  Z-transform 
to 


This  method  was  suggested  by  Prof.  Craig  Comstock,  Cdr. 
USNR,  of  the  Mathematics  Department,  Naval  Postgraduate  School 
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„a,,  .  A   -alkl 
R^(k)  =  e   '  ' 


(F.30) 


since  this  function  is  absolutely  summable. 
The  power  spectrum  of  R  (k)  is 


Sf!(z)  =  Z[R«(k)]  =   Z   e-'^l^lz'^ 

k=-~ 


u 


u 


00  00 

^   -ak  -k  .   -az   ^   -ck  k 
=  Ze    z   +e     Ee    z 


k=0 


k=0 


-a 
e   z 


,     -a  -1    -     -a 
1-ez     1-ez 


(F.31) 


as  long  as  z  is  in  the  annulus  e""  <  jz]  <  e".   Then  by 
collecting  terms 


S°(z)  = 
u 


/  -a    a\ 
z(e    -  e  ) 

(z  -  e   ) (z  -  e  ) 


(F.32) 


Letting  S^'Cz)  be  the  analytic  extension  of  the  power  spectrum 
and  applying  the  inverse  Z-transform,  then  by  residue  theory 


-a  a, 


RCX(k)    =      1-    £  S°'(z)z^  —  =    o^  C  —^ =-^-^ 

u  2iTj  u  z         2ttj         /  „-aN/„ 


(z   -   e      ) (z  -   e    ) 


dz 


-ak 


ctk 


k    >   0 


k   <   0 


=      e 


-a|k| 


(F.33) 
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so  that  this  Is  a  consistent  transform  pair.   Notice  that  in 
the  limit  the  power  spectrum  S^(z)  becomes  the  Dirac  delta 
function  at  z  =  1,  since 


S  (z)  ^  Z[R  (k)]  =  Z[lim  R^Ck)] 

a-*-0 


=   Z   11m  e   '  'z  =  lim  Z  e   '  'z 
k=0  a-'-O  •  a-»-0  k=0 

~  z  =  1 
=  11m  S"(z)  =  =  6  (  z-1)   (F.34) 

a-»-0  0  otherwise 


This  yields  the  transform  pair 


R  (k)  -V  S  (z) 
u       u 


->  6(z-l) 


(F.35a) 


1    k  =  0 
0    k  7^  0. 


--*  1  (F.35b) 


b.   State  Correlations 

It  is  now  possible  to  find  the  correlations  among 
the  states  and  output  for  a  constant  input  autocorrelation, 
In  any  second  order  digital  filter,  there  are  two  transfer 
functions  between  the  input  and  the  states  and  one  between 
input  and  output.   Let  them  be  denoted  by  H, (z),  Hp(z)  and 
H  (z)  respectively. 

From  Appendix  E,  the  power  spectra  for  the  states 
and  output  are  given  by 
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S^j(z)  =  S^(z)H^(l/z)Hj(z)  (F.36) 


where  i,J  =  l,2,v  and  S  (z)  is  the  power  spectrum  of  the 
input.   For  the  case  of  the  highly  correlated  input,  let 


S..(z)  =  lim  S.Vz)  =  H.  (l/z)H.(z)lim  S^'Cz)   (F.37) 

and  consider  S  .(z).   The  corresponding  correlation  function 
is  then 


Ri°^.(k)  =  2iJ  "^  Hjl(1/z)Hj(z)S^(z)z^  ^        (F.38) 


Now  for  any  second  order  filter  the  product  H. (l/z)K . ( z) 
can  be  written  as 


H,(l/z)H,(z)  =  ^ij^^^   (F.39) 

(z-z^) (z-Z2)(z-z^  )(z-Z2  ) 


where  z,  and  z^  are  the  poles  of  the  filter  and  K^  .(z)  is 
a  polynomial  in  z.   Then  the  integrand  in  (F.38)  becomes 


I(z)  =  H.(z)H^.(z)S^(z)z^-^ 

K^_.(z)(e-^  -  e")z^ 


(z-Zj^)  (Z-Z2)  (z-z~  )(z-Z2  )(z-e~  )(z-e  ) 


(F.i]0) 
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For  k  >_  0,  it  Is  possible  to  write  the  partial  expan- 
sion of  (F.40)  as 


A,  (k)    A_(k)  B, (k)   B_(k)  C, (k)    C„(k) 

I(z)  =  (e-°'-e°')[-i +  -i +  -i— y  +  -^  +  -^ +  -? ] 

z-z,     z-z-       -1      -1      -a      a 

1       2  z-z^     z-z^  z-e     z-e 


(F.41) 


where  C,(k)  and  C„(k)  contain  the  factor  (e  °'-e°')  In  their 
denominators  while  the  other  residues  do  not.   Letting 


C^(k)  =  C|(k)/(e~^-e") 


(F.il2) 


then  by  residue  theory 


R^«.(k)  =  [A^(k)+A2(k)](e""-e°')  +  C|(k)   k  >  0 


(F.43) 


so  that 


R.  .(k)  =  11m  R.°(k)  =  11m  C' (k)  =  11m  I(z)(z-e"") 


z=e-« 


11m  e""^H.(eCx)H.(e~") 


H  (1)H  (1) 


k  >  0 


(F.i]i|) 
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It  can  be  shown  that  Equation  (F.^4)  also  holds  for  k  <  0 
except  that  the  last  limit  is 


lim  {e-^^H.  (e")H.(e"°')  +  (e""^  -  e~''^)H,  (-)H  .  (0)} 
a->0       1     J  1    J 


=  H^(1)H^.(1)  (F.45) 


as  long  as  the  filter  has  no  pole  at  z  =  0 . 

The  final  result  is  that  if  the  input  to  a  digital 
filter  has  a  constant  autocorrelation  function  equal  to  N  , 
then  the  auto-  and  cross-correlations  of  the  states  and  output 
is  given  by 


R. .(k)  =  N„H. (l)H.(l)  (F.46) 

ij        0  1     J 


(where  i,j  =  l,2,v),  which  is  also  a  constant  correlation 
function.   It  can  also  be  shown  that  the  cross-correlation 
between  the  input  and  one  of  the  states  or  the  output  is 


R,,(k)  =  N_H,  (1)  =  N„   Z  h.  (n)  (F.iJ?) 


Notice  that  the  cross- correlation  functions  are  even  func- 
tions in  this  case.   Notice  also  that  the  quantity  H.(l)  is 
the  sum  (discrete  Integral)  of  the  unit  pulse  response,  h.(n), 
of  the  associated  state  or  output. 
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APPENDIX  G 

TRANSFORMING  (m)^^  ORDER  COMPLEX  FILTERS 
INTO  (2m)^^  ORDER  REAL  FILTERS 


In  Chapter  V,  the  DFT  Filter  is  an  algorithm  which  con- 
tains first  order  complex  coefficient  digital  filter  sections, 
These  filter  sections  are  purported  to  be  digital  resonators 
at  certain  frequencies  around  the  unit  circle  in  the  Z-plane. 
But  a  resonator  is  expected  to  be  a  second  order  real  coeffi- 
cient algorithm.   It  is  shown  here  that  the  equivalence  is 
justified. 

First  consider  the  first  order  digital  filter  using 
complex  arithmetic  with  one  input  and  one  output  (see  Figure 
G-1): 

x(n)  =  ax(n-l)  +  bu(n)  (G.la) 

y(n)  =  cx(n)  +  du(n)  (G.lb) 

where  a,  b,  c,  d,  u,  x  and  y  are  complex  numbers.  Let 

x(n)  =  x^(n)  +  jx2(n)  (G.2a) 

y(n)  -  y^(n)  +  jygCn)     "  (G.2b) 

u(n)  =  u^(n)  +  jU2(n)  (G.2c) 
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a  =  a^  +  Ja2  (G.2d) 


b  =  b^  +  jb2  (G.2e) 


c  =  c^  +  Jc2  (G.2f) 


d  =  d^  +  jd2  (G.2g) 

where  subscript  (1)  represents  the  real  part  of  the  complex 
number  and  subscript  (2)  Is  the  Imaginary  part.  Then  Equa- 
tions (G.l)  may  be  written 

Xj_(n)  +  jX2(n)  =  a^x^(n-l)  -  a2X-(n-l)  +  jCa-^x-Cn-l) 

+  a2X^(n-l)] 

+  b^u^(n)  -  b2U2(n)  +  j[b^U2(n)  +  b2U^(n)] 

(G.3a) 

y-j^(n)  +  oy2(n)  =  c^x^(n)  -  C2X2(n)  +  j[c^X2(n)  +  C2X-j^(n)] 

+  d,u  (n)  -  d2U2(n)  +  j[d  U2(n)  +  d2U^(n)] 

(G.3b) 

Equating  the  real  and  Im.aglnary  parts  of  Equations  (G.3)  and 
using  state  variable  form  yields: 
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"x^(n)" 

— 

'^1 

-^2 

x^(n-l) 

+ 

\ 

-"2' 

Uj^(n) 

X2(n) 

.^2 

^. 

XjCn-l) 

."2 

"1. 

."2^"' 

(G.ila) 


y^Cn)- 


ygCn) 


"°1 

-^2* 

"x^(n)- 

+ 

r^i 

-■^2" 

"Uj^(n)- 

.^2 

^1. 

X2(n) 

.''2 

<»i. 

_U2(n)_ 

(G.4b) 


These  can  be  recognized  as  the  equations  for  a  second  order 
digital  filter  using  scalar  arithmetic  with  two  Inputs  and 
two  outputs  (see  Figure  G-2) .   The  characteristic  equation 
of  this  second  order  system  Is: 


z^  -  2a^z  +  a^  +  a2  =  z^  -  2Rg[a]z  +  \a\^  (G.5) 


The  pole  locations  are  therefore  z  =  a  and  z  =  a*  where  * 
Indicates  complex  conjugate.   Note  that  If  (a)  Is  a  real 
number  then  ap  =  0  and  a  =  a*  and  this  second  order  filter 
Is  actually  two  uncoupled  first  order  filters  each  with  a 
pole  at  z  =  a  =  a, . 

The  general  result  Is  that  an  m   order  digital  filter 
using  complex  arithmetic  with  p  complex  Inputs  and  q  complex 
outputs  can  be  transformed  into  a  (2m)    order  digital  filter 
using  scalar  arithmetic  with  2p  scalar  inputs  and  2q  scalar 
outputs.   This  is  shown  in  the  following  development. 
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For  the  m   order  complex  filter  we  have  the  equations: 


x(n)  =  Ax(n-l)  +  Bu(n) 


(G.5a) 


y(n)  =  Cx(n)  +  Du(n) 


(G.5b) 


where 


e.g, 


Let 


x(n)  Is  an  m  X  1  column  vector 
u(n)  is  a  p  X  1  column  vector 
y(n)  Is  a  q  X  1  column  vector 
A  Is  an  m  X  m  matrix 
B  Is  an  m  X  p  matrix 
C  is  a  q  X  m  matrix 
D  is  a  q  X  p  matrix 


x(n)  = 


x^(n)' 
XgCn) 


X  (n) 
L  m 


A  = 


^11 


^21 


L  ml 


a 


12 


'22 


Rx,  (n)  be  the  real  part  of  x-,(n) 
Ix-(n)  be  the  imaginary  part  of  x, (n) 
Ra.  ,  be  the  real  part  of  a.  . 


Im 


mm 


(G.6) 
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la. .  be  the  imaginary  part  of  a. 

etc . 
Let  RA  =  (Ra^  )    lA  =  (la^.)    Rx  =  (Rx^^)    etc. 
i.e.  RA  is  the  matrix  whose  elements  are  Ra^ ^ .   Then  equating 
real  and  imaginary  parts  in  (G.5)  and  forming  partitioned 
vectors  and  matrices  as  follows: 


Rx(n) 


Ix(n) 


RA  ;  -lA 


——  — t— — — — 
lA  j   RA 


Rx(n-l) 


Ix(n-l) 


RB 


IB 


-IB 


RB 


Ru(n) 

Iu(n) 
(G.7a) 


Ry(n) 


Iy(n) 


RC 


-IC 


IC  !   RC 


Rx(n) 


Ix(n) 


+ 

RD    !    -ID 

1 

Ru(n) 

1 
ID    I       RD 

Iu(n) 

(G.7b) 


Another  form  of  these  equations  is  found  by  replacing  each 
complex  element  in  each  complex  vector  and  matrix  with  its 
equivalent  scalar  vector  or  scalar  matrix  defined  as 


x^(n) 


RXj^(n) 


Ix^(n) 


^  X^(n) 


(G.8a) 


etc 


'±i 


Ra 


ij 


L""1J 


-la.  . 


Ra 


iJ. 
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-iJ 


(G.8b) 


Then  Equations  (G.5)  become 


X^(n) 


XjCn) 


^11 


— m 


A2I 


A12I 


-22 


A   ' 


4.  - 


— Im 


A 
-mm 


X^(n-l) 


X2(n-1) 


X  (n- 

— m 


^11 1  ^12 

___  J 


^21 1  ^22 

I j._ 

.1   .1 


I 

t 


5nl 


I 
I 

'B 

•  -mp 


Ul(n) 


(G.9a) 


Y^(n) 


y^(n) 


_5-ll|  ^12!  • 

C   I  "''''" 
-21 1 

r- 


I 

1  n 

'I  -Im 
^ 


qm 


X^(n) 


^11 


^21 


D^  !... 


D 


-1- 
I 


qi! 


D 


Ip 


.  "d 

I  qp 


U  (n) 

-P 


(G.9b) 


J 


Equations  G.7a  and  G.9a  each  represent  2m  real  state  equations 
with  2p  real  inputs.   Equations  G.7b  and  G.9b  each  represent 
2q  real  output  equations  with  2p  real  inputs. 
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x(n) 


u(n) 


-& 


<i>-^ 


-<i> 


x{n-l) 

— (3> 


Figure  G-1 .     First  Order  Complex  Coefficient  Digital   Filter 


Figure  G-2.     Second  Order  Real   Coefficient  Digital   Filter 
Equivalent  to  Figure  G-1. 
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APPENDIX  H 

COMPLEX  INTEGRATION  INVOLVING  A 
COMPLEX  DIRAC  DELTA  FUNCTION 


From  the  results  of  Appendix  F  it  is  possible  to  propose 
the  following  theorem. 

THEOREM:   If  H(z)  is  an  analytic  function  on  the  unit  circle, 
then  if  H(z)  has  only  simple  poles  inside  the  unit 
circle 


t>   H(z)  6(z-l)  —  =  27rjH(l)  (H.l) 


where  the  Integral  is  taken  counter-clockwise  around  the  unit 
circle  and  5(z-l)  is  a  Dirac  delta  function  defined  by 


6(z-l)  ^  lim   -^^ --^1-     =  (H.2) 

a->-0^   (z-e"°')(z-e")      0    z  7^  1 


PROOF:     Consider  the  Integral 


1(a)  =  ^  H(z)(e-^-e")   ^^       ^^^        (^,^3) 
(z-e-^)(z-e^) 


If  H(z)  has  k  simple  poles  inside  the  unit  circle,  then  by 
residue  theory 


k 
1(a)  =  2Trj(e~"-e°)   L  R.  +  27rjH(e"")  (H.^l) 

1  =  1  ^ 
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where  R.  Is  the  residue  of  ^±11 ^t  the  i^^  pole 

(z-e   )(z-e  ) 
of  H(z)  Inside  the  unit  circle.   Notice  that  R.  does  not 

have  the  factor  (e   +e  )  in  Its  denominator  unless  H(z) 

has  a  pole  at  e"  .   To  avoid  any  conflict  assume  that  the 

pole  with  the  largest  modulus  inside  the  unit  circle  has 

|z|  =  b,  and  restrict  a  such  that  0  <  a  <  -In  b.   Under 

this  condition 


lim  1(a)  =  2TTJH(1)  (H.5) 

a-»-0 


On  the  other  hand,  by  (H.3) 


lim  1(a)  =  6   H(z)6(z-1)  ^  (H.6) 

a->-0  ^ 


QED 

The  previous  theorem  leads  to  the  hypothesis  that  if 
^   is  on  the  closed  contour  C,  and  H(z)  is  analytic  on  C  and  has 
only  simple  poles  Inside  C,  then 


(5  H(z)6(z-iJ;)  ^=  27rjH(i|i)  (H.7) 


where  the  integral  is  taken  counter-clockwise  around  C 
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